-- Logs begin at Mon 2022-09-26 09:50:57 UTC, end at Mon 2022-09-26 10:53:46 UTC. -- Sep 26 10:18:12.724665 dv-cinder-844563-7-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 26 10:18:12.733594 dv-cinder-844563-7-iscsi devstack@keystone.service[81412]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 26 10:18:12.735597 dv-cinder-844563-7-iscsi devstack@keystone.service[81412]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Sep 26 10:18:12 2022] *** Sep 26 10:18:12.735597 dv-cinder-844563-7-iscsi devstack@keystone.service[81412]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 26 10:18:12.735597 dv-cinder-844563-7-iscsi devstack@keystone.service[81412]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 26 10:18:12.735597 dv-cinder-844563-7-iscsi devstack@keystone.service[81412]: nodename: dv-cinder-844563-7-iscsi Sep 26 10:18:12.735597 dv-cinder-844563-7-iscsi devstack@keystone.service[81412]: machine: x86_64 Sep 26 10:18:12.735597 dv-cinder-844563-7-iscsi devstack@keystone.service[81412]: clock source: unix Sep 26 10:18:12.735597 dv-cinder-844563-7-iscsi devstack@keystone.service[81412]: pcre jit disabled Sep 26 10:18:12.735597 dv-cinder-844563-7-iscsi devstack@keystone.service[81412]: detected number of CPU cores: 4 Sep 26 10:18:12.735597 dv-cinder-844563-7-iscsi devstack@keystone.service[81412]: current working directory: / Sep 26 10:18:12.735597 dv-cinder-844563-7-iscsi devstack@keystone.service[81412]: detected binary path: /usr/bin/uwsgi-core Sep 26 10:18:12.735597 dv-cinder-844563-7-iscsi devstack@keystone.service[81412]: your processes number limit is 47680 Sep 26 10:18:12.735597 dv-cinder-844563-7-iscsi devstack@keystone.service[81412]: your memory page size is 4096 bytes Sep 26 10:18:12.736243 dv-cinder-844563-7-iscsi devstack@keystone.service[81412]: detected max file descriptor number: 2048 Sep 26 10:18:12.736243 dv-cinder-844563-7-iscsi devstack@keystone.service[81412]: lock engine: pthread robust mutexes Sep 26 10:18:12.736243 dv-cinder-844563-7-iscsi devstack@keystone.service[81412]: thunder lock: enabled Sep 26 10:18:12.736243 dv-cinder-844563-7-iscsi devstack@keystone.service[81412]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 26 10:18:12.736243 dv-cinder-844563-7-iscsi devstack@keystone.service[81412]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 26 10:18:12.774577 dv-cinder-844563-7-iscsi devstack@keystone.service[81412]: Python main interpreter initialized at 0x55cca0b4e9c0 Sep 26 10:18:12.774577 dv-cinder-844563-7-iscsi devstack@keystone.service[81412]: python threads support enabled Sep 26 10:18:12.774577 dv-cinder-844563-7-iscsi devstack@keystone.service[81412]: your server socket listen backlog is limited to 100 connections Sep 26 10:18:12.774577 dv-cinder-844563-7-iscsi devstack@keystone.service[81412]: your mercy for graceful operations on workers is 90 seconds Sep 26 10:18:12.774942 dv-cinder-844563-7-iscsi devstack@keystone.service[81412]: mapped 403077 bytes (393 KB) for 2 cores Sep 26 10:18:12.775031 dv-cinder-844563-7-iscsi devstack@keystone.service[81412]: *** Operational MODE: preforking *** Sep 26 10:18:12.775253 dv-cinder-844563-7-iscsi devstack@keystone.service[81412]: *** uWSGI is running in multiple interpreter mode *** Sep 26 10:18:12.775253 dv-cinder-844563-7-iscsi devstack@keystone.service[81412]: spawned uWSGI master process (pid: 81412) Sep 26 10:18:12.775452 dv-cinder-844563-7-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 26 10:18:12.778253 dv-cinder-844563-7-iscsi devstack@keystone.service[81412]: spawned uWSGI worker 1 (pid: 81420, cores: 1) Sep 26 10:18:12.778253 dv-cinder-844563-7-iscsi devstack@keystone.service[81412]: spawned uWSGI worker 2 (pid: 81421, cores: 1) Sep 26 10:18:12.778253 dv-cinder-844563-7-iscsi devstack@keystone.service[81412]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 26 10:18:17.026002 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 26 10:18:17.026002 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: from cryptography.utils import int_from_bytes Sep 26 10:18:17.026827 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 26 10:18:17.026827 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: from cryptography.utils import int_from_bytes Sep 26 10:18:17.026945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 26 10:18:17.026945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: from cryptography.utils import int_from_bytes Sep 26 10:18:17.026945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 26 10:18:17.026945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: from cryptography.utils import int_from_bytes Sep 26 10:18:17.077529 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 26 10:18:17.077678 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 26 10:18:17.078048 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 26 10:18:17.078148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] command line args: [] {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 26 10:18:17.078229 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 26 10:18:17.078330 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] command line args: [] {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 26 10:18:17.078413 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 26 10:18:17.078538 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ================================================================================ {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 26 10:18:17.078622 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 26 10:18:17.078756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ================================================================================ {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 26 10:18:17.078839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] admin_token = **** {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.079005 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] admin_token = **** {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.079094 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] config_dir = [] {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.079199 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.079276 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] config_dir = [] {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.079383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] config_source = [] {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.079459 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.079565 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] debug = True {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.079654 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] config_source = [] {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.079758 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] debug = True {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.079835 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.080073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.080212 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.080317 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.080420 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.080521 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.080616 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.080724 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.080829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.080926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.081021 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] list_limit = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.081115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] list_limit = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.081215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] log_config_append = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.081310 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] log_config_append = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.081416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.081518 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.081619 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] log_dir = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.081714 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] log_dir = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.081809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] log_file = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.081903 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] log_file = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.082014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.082156 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.082242 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.082364 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.082439 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.082553 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.082636 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.082710 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.082829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.082911 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.082982 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.083062 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.083139 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.083215 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.083377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.083461 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.083587 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.083669 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.083823 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.083969 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.084051 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.084128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.084275 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.084402 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.084525 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.084651 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.084771 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.084884 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.085003 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.085126 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.085466 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.085590 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.085717 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] publish_errors = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.085822 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] publish_errors = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.085933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.086076 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.086194 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.086317 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.086429 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.086541 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.086658 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.086792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.086915 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.087029 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.087148 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] standard_threads = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.087270 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] standard_threads = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.087383 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.087504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.087617 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.087736 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.087849 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.087961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.088074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] use_journal = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.088249 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] use_journal = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.088372 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] use_json = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.088492 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] use_json = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.088606 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] use_stderr = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.088718 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] use_stderr = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.088831 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] use_syslog = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.088951 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] use_syslog = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.089071 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.089186 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 10:18:17.089299 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.089417 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.089531 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.089652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.089766 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.089884 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.089997 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.090125 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.090238 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.090370 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.090485 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.090598 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.090716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.090829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.090940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.091060 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.091685 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.091798 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.091910 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.092028 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.092141 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.092342 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.092479 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.092596 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.092715 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.092828 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.092955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.093081 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.093196 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.093316 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.093429 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.093542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.093654 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.093775 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.093895 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.094009 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.094139 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.094262 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.094378 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.094561 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.094677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.094790 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.094902 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.095014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.095137 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.095251 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.095364 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.095489 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.095603 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.095716 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.095836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.095950 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.096063 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.096180 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.096294 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.096406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.096527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.096640 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.096752 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.096864 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.096982 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.097101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.097226 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.097340 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.097452 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.097571 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.097685 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.097797 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.097917 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.098045 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.098161 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.098273 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.098385 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.098496 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.098625 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.098740 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.098857 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.098970 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.099083 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.099195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.099315 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.099429 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.099541 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.099655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.099767 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.099878 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.100068 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.100184 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.100296 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.100421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.100542 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.100661 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.100782 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.100896 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.101008 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.101120 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.101232 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.101345 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.101469 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.101583 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.101696 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.101815 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.101928 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.102054 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.102181 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.102280 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.102375 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.102470 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.102569 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.102670 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.102775 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.102871 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.102966 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.103095 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.103201 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.103329 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.103444 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.103556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.103669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.103782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.103885 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.104008 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.104123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.104236 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.104349 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.104470 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.104593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.104716 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.104831 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.104956 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.105061 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.105179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.105300 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.105416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.105529 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.105653 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.105768 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.105882 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.105992 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.106143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.106257 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.106376 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.106497 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.106617 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.106763 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.107408 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.107526 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.107648 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.107759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.107873 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.107993 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.108113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.108227 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.108347 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.108462 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.108575 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.108688 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.108801 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.108914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.109040 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.109155 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.109268 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.109381 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.109494 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.109606 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.109727 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.109842 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.109955 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.110083 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.110199 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.110322 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.110445 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.110560 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.110674 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.110787 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.110901 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.111014 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.111136 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.111252 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.111365 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.111478 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.111591 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.111704 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.111887 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.112010 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.112130 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.112246 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.112359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.112482 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.112606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.112729 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.112844 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.112957 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.113070 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.113191 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.113309 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.113432 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.113547 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.113661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.113780 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.113895 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.114009 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.114151 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.114269 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.114383 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.114496 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.114610 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.114730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.114853 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.114968 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.115088 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.115209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.115332 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.115332 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] auth.external = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.115593 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.115708 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.115831 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] auth.external = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.115831 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.116047 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.116047 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.116263 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.116387 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] auth.password = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.116510 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.116632 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] auth.token = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.116747 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] auth.password = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.116861 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.116982 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] auth.token = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.117097 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.117219 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.117338 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.117454 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.117568 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.117683 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.117797 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.117919 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.118093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.118212 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.118326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.118440 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.118565 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] credential.caching = True {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.118689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.118811 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.118926 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] credential.caching = True {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.119039 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.119154 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.119268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.119390 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.119506 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.119621 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.119735 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.119848 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.119962 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.120084 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.120205 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.120321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.120436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.120549 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.120664 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.120787 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.120904 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.121018 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.121132 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.121246 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.121360 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.121481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.121597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.121712 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.121826 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.121939 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.122067 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.122199 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.122316 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] federation.caching = True {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.122431 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.122545 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.122660 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] federation.caching = True {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.122774 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.122897 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.123012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.123132 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.123247 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.123361 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.123475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.123597 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.123714 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.123828 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.123942 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.124056 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.124178 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.124303 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.124417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.124532 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.124645 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.124765 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.124889 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.125004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.125118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] identity.caching = True {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.125232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.125346 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.125460 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] identity.caching = True {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.125647 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.125765 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.125881 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.125995 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.126151 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.126266 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.126392 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.126509 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.126623 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.126737 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.126851 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.126966 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.127088 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.127204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.127322 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.127437 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.127542 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.127655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.127784 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.127902 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.128016 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.128131 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.128245 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.128360 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.128482 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.128597 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.128711 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.128824 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.128939 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.129054 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.129179 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.129295 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.129409 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.129524 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.129637 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.129751 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.129873 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.129988 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.130120 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.130234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.130348 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.130472 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.130596 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.130712 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.130826 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.130940 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.131054 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.131168 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.131290 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.131406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.131520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.131634 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.131748 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.131862 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.131984 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.132100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.132214 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.132329 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.132443 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.132558 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.132680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.132801 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.132917 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.133031 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.133146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.133268 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.133383 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.133497 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.133611 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.133725 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.133839 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.133961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.134136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.134252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.134366 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.134481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.134595 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.134723 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.134840 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.134955 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.135068 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.135182 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.135297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.135419 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.135535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.135535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.135750 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.135750 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.135964 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.136085 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.136201 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.136317 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.136430 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.136544 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.136657 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.136657 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.136880 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.137002 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.user = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.137118 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.137232 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.137350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.137350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.user = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.137575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.137689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.137803 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.137803 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.138033 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.138033 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.138266 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.138380 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.138495 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.138495 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.138709 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.138831 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.138947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.139061 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.139175 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.139288 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.139403 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.139526 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.139642 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.139756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.139870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.139983 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.140103 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.140226 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.140341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.140457 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.140571 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.140685 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.140799 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.140921 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.141036 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.141150 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.141263 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.141377 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.141491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.141614 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.141730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.141844 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.141958 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.142111 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.142228 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.142354 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.142471 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.142586 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.142701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.142814 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.142928 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.143051 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.143167 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.143282 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.143396 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.143510 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.143624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.143746 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.143862 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.143976 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.144091 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.144205 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.144319 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.144442 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.144557 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.144672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.144786 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.144900 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.145015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.145137 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.145253 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.145373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.145488 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.145602 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] resource.caching = True {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.145716 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.145839 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.145954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.146080 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.146196 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] resource.caching = True {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.146310 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.146425 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.146550 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.146668 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.146783 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.146904 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.147020 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.147135 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.147257 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.147376 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.147491 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.147605 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.147719 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.147904 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.148022 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] role.caching = True {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.148136 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.148249 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] role.driver = sql {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.148370 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.148486 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.148609 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] role.caching = True {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.148725 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.148839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] role.driver = sql {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.148954 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.149068 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.149181 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.149303 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.149419 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.149533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.149646 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.149760 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.149873 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.149997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.150131 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.150246 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.150360 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.150473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.150587 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.150712 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.150828 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.150942 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.151057 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.151171 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.151285 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.151407 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.151523 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.151637 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.151637 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.151852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.151965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.152089 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.152089 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.152304 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.152418 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.152531 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.152644 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.152766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.152881 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.152995 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.153110 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.153224 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.153350 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.153466 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.154153 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.154286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.154402 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.154517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.154640 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.154755 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.154869 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.154984 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.155098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.155214 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.155336 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.155451 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.155565 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.155679 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.155793 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.155907 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.156028 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.156144 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.156259 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.156373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.156487 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.156601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.156723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] token.caching = True {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.156838 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.156952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.157067 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.157180 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.157299 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] token.caching = True {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.157422 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.157538 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.157649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.157754 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.157851 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.157966 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.158114 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.158232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.158348 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.158463 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.158577 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.158691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.158813 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.158929 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.159043 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.159158 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.159271 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.159386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.159509 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.159624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.159738 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.159852 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.159966 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.160080 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.160202 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.160327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.160451 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.160567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.160682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.160796 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.160919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] database.connection = **** {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.161043 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.161158 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.161352 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.161498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.161613 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] database.connection = **** {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.161727 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.161727 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.161941 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.162083 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.162201 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.162316 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.162430 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.162546 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.162675 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.162799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.162915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.163029 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.163143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.163257 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.163371 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.163493 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.163610 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.163724 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.163838 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.164010 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.164160 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.164285 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.164401 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.164516 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.164632 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.164746 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.164861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.164983 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.165099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.165213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.165327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.165442 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 26 10:18:17.165556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.165556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.165556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 10:18:17.165556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81420) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 26 10:18:17.191698 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.191925 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.192147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.192369 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.192599 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.192826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.200417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.201161 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.229138 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.229365 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.235779 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.236095 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.284001 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.285097 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.332402 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.332574 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.333013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.333245 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.487589 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.487806 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.487898 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.488022 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.488104 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.488238 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.488320 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.488450 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.488531 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.488660 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.488750 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.488871 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.511875 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.512051 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.512180 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.512263 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.512358 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.512491 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.512574 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.512716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.512801 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.512936 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.513724 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.513938 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.514555 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.514790 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.525458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.531034 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.538561 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.538793 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.539018 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.539242 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.539459 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.539672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.539881 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.543444 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.543675 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.543896 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.544113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.544354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.544568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.544781 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81420) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 10:18:17.556370 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.556580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 26 10:18:17.556891 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.556975 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.557103 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 26 10:18:17.557185 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 26 10:18:17.557371 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.557454 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.557628 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 26 10:18:17.557710 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.557882 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.557972 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.558167 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.558250 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.558423 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.558681 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.567745 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.568021 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.568289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.568373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.568569 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.568653 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.568839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.569123 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.576072 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.576666 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.579264 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.579547 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.579829 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.580113 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.580394 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.580655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.580924 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.581197 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.581466 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.581625 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.581736 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.581911 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.582001 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.582218 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.582548 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.582793 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.583055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.583327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.583600 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.583869 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.584143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.584411 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.605029 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.606508 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.606817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.607579 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.609041 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.609328 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.611447 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.611731 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.612009 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.613999 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.614310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.614593 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.618328 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.618614 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.620915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.621202 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.624679 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.624970 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.625249 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.625529 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.625801 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.626105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.626376 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.626644 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.627328 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.627619 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.627907 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.628197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.628475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.628751 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.629023 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.629293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.642634 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.642910 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.645286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.645565 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.646885 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.649423 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.650144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.650437 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.652690 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.652989 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.654670 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.657229 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.657810 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.660445 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.661000 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.661292 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.661578 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.661857 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.662165 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.662454 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.663700 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.663999 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.664284 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.664569 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.664857 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.665138 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.678163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.678449 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.678714 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.678972 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.680961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.681249 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.681516 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.681784 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.686310 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.687845 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.689211 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.689354 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.689627 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.690766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.692302 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.692390 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.692578 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.692672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.692908 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.693195 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.693584 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.695365 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.695652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.695934 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.696216 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.696611 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.705451 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.705732 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.706011 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.706313 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.706700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.706981 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.707263 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.708623 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.708916 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.709201 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.709483 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.709863 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.710177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.710467 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.722592 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.725482 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.725849 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.728371 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.728788 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.729895 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.731375 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.731656 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.731738 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.732070 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.733163 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.734655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.734936 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.735333 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.738404 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.739855 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.741675 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.742747 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.743146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.743229 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.743431 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.743705 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.746037 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.746439 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.746727 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.747008 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.752000 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.752288 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.752568 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.755328 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.755617 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.755897 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.759412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.759697 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.759978 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.760254 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.760530 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.760812 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.761085 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.761350 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.761631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.761911 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.762223 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.762485 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.762747 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.762837 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81420) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 10:18:17.763597 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.763693 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.763884 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.764160 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.764435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.764707 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.764978 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.765254 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.765525 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.765801 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.766107 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.766372 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.766646 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.766932 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81420) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 10:18:17.797231 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55cca0b4e9c0 pid: 81421 (default app) Sep 26 10:18:17.800643 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a8204-3b19-4410-aa98-f356e78d4c84 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:17.801081 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a8204-3b19-4410-aa98-f356e78d4c84 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:17.801357 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a8204-3b19-4410-aa98-f356e78d4c84 None None] PATH_INFO: `/v3` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:17.801628 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55cca0b4e9c0 pid: 81420 (default app) Sep 26 10:18:17.808489 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1/1] 10.222.0.49 () {50 vars in 791 bytes} [Mon Sep 26 10:18:17 2022] GET /identity/v3/ => generated 255 bytes in 11 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:24.566033 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a25895-9e6f-45e0-837c-e2a40a0fa6b7 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:24.566546 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a25895-9e6f-45e0-837c-e2a40a0fa6b7 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:24.566649 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a25895-9e6f-45e0-837c-e2a40a0fa6b7 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:24.570875 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1/2] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:18:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:18:24.577061 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01346d7a-fdae-4fb8-96f7-1552721f218f None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:24.577616 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01346d7a-fdae-4fb8-96f7-1552721f218f None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:24.577616 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01346d7a-fdae-4fb8-96f7-1552721f218f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:24.647570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: INFO dbcounter [None req-01346d7a-fdae-4fb8-96f7-1552721f218f None None] Registered counter for database keystone Sep 26 10:18:24.666115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG oslo_db.sqlalchemy.engines [None req-01346d7a-fdae-4fb8-96f7-1552721f218f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81421) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 26 10:18:24.672510 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writer thread running {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 26 10:18:24.857230 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG passlib.handlers.bcrypt [None req-01346d7a-fdae-4fb8-96f7-1552721f218f None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81421) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 26 10:18:24.857545 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG passlib.handlers.bcrypt [None req-01346d7a-fdae-4fb8-96f7-1552721f218f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81421) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 26 10:18:24.916228 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-01346d7a-fdae-4fb8-96f7-1552721f218f None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:18:24.917904 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-01346d7a-fdae-4fb8-96f7-1552721f218f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:18:25.046996 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 2/3] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:18:24 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 471 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 26 10:18:25.520510 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a25895-9e6f-45e0-837c-e2a40a0fa6b7 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:25.590271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: INFO dbcounter [None req-c4a25895-9e6f-45e0-837c-e2a40a0fa6b7 None None] Registered counter for database keystone Sep 26 10:18:25.611870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG oslo_db.sqlalchemy.engines [None req-c4a25895-9e6f-45e0-837c-e2a40a0fa6b7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81420) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 26 10:18:25.618296 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writer thread running {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 26 10:18:25.739894 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a25895-9e6f-45e0-837c-e2a40a0fa6b7 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:25.744215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23440f-630b-4ef5-8ea3-6bbff4c7c18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:25.744898 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23440f-630b-4ef5-8ea3-6bbff4c7c18f admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:25.745174 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23440f-630b-4ef5-8ea3-6bbff4c7c18f admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:25.745444 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23440f-630b-4ef5-8ea3-6bbff4c7c18f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:25.754720 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23440f-630b-4ef5-8ea3-6bbff4c7c18f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:25.807718 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6b23440f-630b-4ef5-8ea3-6bbff4c7c18f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 26 10:18:25.808075 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23440f-630b-4ef5-8ea3-6bbff4c7c18f admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:25.818364 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 2/4] 10.222.0.49 () {58 vars in 1324 bytes} [Mon Sep 26 10:18:25 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 26 10:18:25.824219 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01346d7a-fdae-4fb8-96f7-1552721f218f None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:25.842236 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01346d7a-fdae-4fb8-96f7-1552721f218f None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:25.846683 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3718b2f-b2ac-463b-b547-3727f4215a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:25.847819 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3718b2f-b2ac-463b-b547-3727f4215a11 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:25.848095 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3718b2f-b2ac-463b-b547-3727f4215a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:25.848375 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3718b2f-b2ac-463b-b547-3727f4215a11 admin admin] PATH_INFO: `/v3/projects/9cef67c15d1f4b9695af7c22c68b5822` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:25.849991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3718b2f-b2ac-463b-b547-3727f4215a11 admin admin] RBAC: Authorizing `identity:get_project(project_id=9cef67c15d1f4b9695af7c22c68b5822)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:25.902615 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c3718b2f-b2ac-463b-b547-3727f4215a11 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 26 10:18:25.902961 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3718b2f-b2ac-463b-b547-3727f4215a11 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:25.907477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 3/5] 10.222.0.49 () {56 vars in 1207 bytes} [Mon Sep 26 10:18:25 2022] GET /identity/v3/projects/9cef67c15d1f4b9695af7c22c68b5822 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:28.039173 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9430100-4823-4ed3-91a1-d6dd63123f26 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:28.039681 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9430100-4823-4ed3-91a1-d6dd63123f26 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:28.039764 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9430100-4823-4ed3-91a1-d6dd63123f26 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:28.043883 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 3/6] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:18:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:18:28.049240 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba119f-592c-4774-8266-d12cdd89cbd3 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:28.049598 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba119f-592c-4774-8266-d12cdd89cbd3 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:28.049788 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba119f-592c-4774-8266-d12cdd89cbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:28.069481 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-a8ba119f-592c-4774-8266-d12cdd89cbd3 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:18:28.070960 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-a8ba119f-592c-4774-8266-d12cdd89cbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:18:28.079791 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 4/7] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:18:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 26 10:18:28.345744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9430100-4823-4ed3-91a1-d6dd63123f26 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:28.357972 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9430100-4823-4ed3-91a1-d6dd63123f26 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:28.362197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba35d6f-ce0b-4b31-8da2-748066401da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:28.362788 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba35d6f-ce0b-4b31-8da2-748066401da7 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:28.363065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba35d6f-ce0b-4b31-8da2-748066401da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:28.363338 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba35d6f-ce0b-4b31-8da2-748066401da7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:28.370889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba35d6f-ce0b-4b31-8da2-748066401da7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:28.373150 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba35d6f-ce0b-4b31-8da2-748066401da7 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:28.382140 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 4/8] 10.222.0.49 () {58 vars in 1324 bytes} [Mon Sep 26 10:18:28 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 10:18:28.387207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba119f-592c-4774-8266-d12cdd89cbd3 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:28.396847 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba119f-592c-4774-8266-d12cdd89cbd3 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:28.401097 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e80f3-26d9-4991-9279-f677130ab6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:28.401711 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e80f3-26d9-4991-9279-f677130ab6f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:28.401987 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e80f3-26d9-4991-9279-f677130ab6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:28.402289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e80f3-26d9-4991-9279-f677130ab6f5 admin admin] PATH_INFO: `/v3/users/7ecff1fbd79248b7bcd1e69035be3515` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:28.403583 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e80f3-26d9-4991-9279-f677130ab6f5 admin admin] RBAC: Authorizing `identity:get_user(user_id=7ecff1fbd79248b7bcd1e69035be3515)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:28.405564 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e80f3-26d9-4991-9279-f677130ab6f5 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:28.407098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 5/9] 10.222.0.49 () {56 vars in 1198 bytes} [Mon Sep 26 10:18:28 2022] GET /identity/v3/users/7ecff1fbd79248b7bcd1e69035be3515 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:30.596513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e6566-52d2-4540-a8fe-f6c6de291655 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:30.597094 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e6566-52d2-4540-a8fe-f6c6de291655 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:30.597094 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e6566-52d2-4540-a8fe-f6c6de291655 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:30.598400 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 5/10] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:18:30.603455 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7774092-e3db-47a6-8f33-3913a69bacc9 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:30.603848 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7774092-e3db-47a6-8f33-3913a69bacc9 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:30.604001 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7774092-e3db-47a6-8f33-3913a69bacc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:30.622178 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-f7774092-e3db-47a6-8f33-3913a69bacc9 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:18:30.623383 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-f7774092-e3db-47a6-8f33-3913a69bacc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:18:30.666671 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c200e-9cee-4b92-b25e-99d3e5a3a0dd None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:30.666944 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c200e-9cee-4b92-b25e-99d3e5a3a0dd None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:30.667216 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c200e-9cee-4b92-b25e-99d3e5a3a0dd None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:30.668596 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 6/11] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:18:30.674458 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60a1ad-f0be-4c0f-9502-7a86a38141f1 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:30.674741 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60a1ad-f0be-4c0f-9502-7a86a38141f1 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:30.675018 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60a1ad-f0be-4c0f-9502-7a86a38141f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:30.703081 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 6/12] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:18:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 10:18:30.717858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG passlib.handlers.bcrypt [None req-9d60a1ad-f0be-4c0f-9502-7a86a38141f1 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81420) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 26 10:18:30.718266 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG passlib.handlers.bcrypt [None req-9d60a1ad-f0be-4c0f-9502-7a86a38141f1 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81420) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 26 10:18:30.767504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-9d60a1ad-f0be-4c0f-9502-7a86a38141f1 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:18:30.769048 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-9d60a1ad-f0be-4c0f-9502-7a86a38141f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:18:30.868355 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 7/13] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:18:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 10:18:30.965556 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7774092-e3db-47a6-8f33-3913a69bacc9 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:30.989652 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:31.068620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7774092-e3db-47a6-8f33-3913a69bacc9 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:31.101259 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b20acb-e64a-40cd-bd7b-b5d3264d5746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:31.101886 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b20acb-e64a-40cd-bd7b-b5d3264d5746 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:31.102191 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b20acb-e64a-40cd-bd7b-b5d3264d5746 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:31.102463 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b20acb-e64a-40cd-bd7b-b5d3264d5746 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:31.114179 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b20acb-e64a-40cd-bd7b-b5d3264d5746 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:18:31.123539 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-89b20acb-e64a-40cd-bd7b-b5d3264d5746 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:18:31.123539 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:18:31.123539 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:18:31.123539 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:18:31.123539 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:18:31.123539 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:18:31.123539 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:18:31.123539 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:18:31.123539 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:18:31.123539 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:18:31.123539 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:18:31.123539 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:18:31.123539 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:18:31.123539 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:18:31.123539 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:18:31.123539 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:18:31.124591 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:18:31.124591 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:18:31.124591 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:18:31.124591 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:18:31.124591 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:18:31.124591 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:18:31.124591 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:18:31.124591 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:18:31.124591 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:18:31.124591 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:18:31.124591 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:18:31.124591 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:18:31.124591 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:18:31.124591 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:18:31.124591 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:18:31.124591 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:18:31.125439 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:18:31.125439 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:18:31.125439 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:18:31.125439 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:18:31.125439 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:18:31.125439 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:18:31.125439 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:18:31.125439 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:18:31.125439 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:18:31.125439 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:18:31.125439 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:18:31.125439 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:18:31.125439 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 7/14] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:18:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:18:31.130410 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60a1ad-f0be-4c0f-9502-7a86a38141f1 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:31.138398 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b20acb-e64a-40cd-bd7b-b5d3264d5746 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:31.219796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:31.231460 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60a1ad-f0be-4c0f-9502-7a86a38141f1 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:31.234360 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:31.238698 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b20acb-e64a-40cd-bd7b-b5d3264d5746 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:31.268849 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0019b-e81a-4af0-9b6f-bf0c9bff3d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:31.269708 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0019b-e81a-4af0-9b6f-bf0c9bff3d7b None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:31.269997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0019b-e81a-4af0-9b6f-bf0c9bff3d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:31.270297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0019b-e81a-4af0-9b6f-bf0c9bff3d7b None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:31.271127 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb0019b-e81a-4af0-9b6f-bf0c9bff3d7b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:18:31.271226 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2757f87-d50a-4c2c-ad5a-566031dc4569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:31.271429 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb0019b-e81a-4af0-9b6f-bf0c9bff3d7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:31.271833 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2757f87-d50a-4c2c-ad5a-566031dc4569 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:31.272111 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2757f87-d50a-4c2c-ad5a-566031dc4569 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:31.272380 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2757f87-d50a-4c2c-ad5a-566031dc4569 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:31.272867 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb0019b-e81a-4af0-9b6f-bf0c9bff3d7b None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:31.285120 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 8/15] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:18:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:31.290744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0019b-e81a-4af0-9b6f-bf0c9bff3d7b None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:31.338751 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2757f87-d50a-4c2c-ad5a-566031dc4569 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:31.340213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2757f87-d50a-4c2c-ad5a-566031dc4569 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:31.389482 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0019b-e81a-4af0-9b6f-bf0c9bff3d7b None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:31.418062 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:31.424554 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfaab41-c8b4-417a-8b0a-54d77dca6fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:31.425404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfaab41-c8b4-417a-8b0a-54d77dca6fb3 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:31.425680 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfaab41-c8b4-417a-8b0a-54d77dca6fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:31.425954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfaab41-c8b4-417a-8b0a-54d77dca6fb3 None admin] PATH_INFO: `/v3/users/7ecff1fbd79248b7bcd1e69035be3515` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:31.427452 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfaab41-c8b4-417a-8b0a-54d77dca6fb3 None admin] RBAC: Authorizing `identity:get_user(user_id=7ecff1fbd79248b7bcd1e69035be3515)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:31.428897 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfaab41-c8b4-417a-8b0a-54d77dca6fb3 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:31.430504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 9/16] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:18:31 2022] GET /identity/v3/users/7ecff1fbd79248b7bcd1e69035be3515 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:18:31.435792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfaab41-c8b4-417a-8b0a-54d77dca6fb3 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:31.437920 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 8/17] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:18:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 10:18:31.443113 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2757f87-d50a-4c2c-ad5a-566031dc4569 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:31.471251 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:31.537536 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfaab41-c8b4-417a-8b0a-54d77dca6fb3 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:31.540558 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2757f87-d50a-4c2c-ad5a-566031dc4569 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:31.574130 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9d78d-284e-44ac-bc76-96f4b87ec399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:31.574759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9d78d-284e-44ac-bc76-96f4b87ec399 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:31.575039 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9d78d-284e-44ac-bc76-96f4b87ec399 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:31.575287 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb0e08-f813-44f6-ba23-332c1c03be83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:31.575407 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9d78d-284e-44ac-bc76-96f4b87ec399 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:31.576440 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb0e08-f813-44f6-ba23-332c1c03be83 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:31.576715 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb0e08-f813-44f6-ba23-332c1c03be83 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:31.576984 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb0e08-f813-44f6-ba23-332c1c03be83 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:31.578119 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb0e08-f813-44f6-ba23-332c1c03be83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:31.579831 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb0e08-f813-44f6-ba23-332c1c03be83 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:31.581389 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 10/18] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:18:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:18:31.586474 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f9d78d-284e-44ac-bc76-96f4b87ec399 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:31.587043 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb0e08-f813-44f6-ba23-332c1c03be83 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:31.587928 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f9d78d-284e-44ac-bc76-96f4b87ec399 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:31.605206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-a2f9d78d-284e-44ac-bc76-96f4b87ec399 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:18:31.605206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:18:31.605206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:18:31.605206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:18:31.605206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:18:31.605206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:18:31.605206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:18:31.605206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:18:31.605206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:18:31.605206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:18:31.605206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:18:31.605206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:18:31.605206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:18:31.605206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:18:31.605206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:18:31.605206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:18:31.606690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:18:31.606690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:18:31.606690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:18:31.606690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:18:31.606690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:18:31.606690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:18:31.606690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:18:31.606690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:18:31.606690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:18:31.606690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:18:31.606690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:18:31.606690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:18:31.606690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:18:31.606690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:18:31.606690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:18:31.606690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:18:31.606690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:18:31.607516 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:18:31.607516 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:18:31.607516 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:18:31.607516 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 9/19] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:18:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:18:31.611854 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9d78d-284e-44ac-bc76-96f4b87ec399 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:31.646162 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:31.687532 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb0e08-f813-44f6-ba23-332c1c03be83 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:31.689033 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:31.712797 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9d78d-284e-44ac-bc76-96f4b87ec399 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:31.724207 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8722dc7-f448-4744-8613-b4b33e9f8ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:31.724812 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8722dc7-f448-4744-8613-b4b33e9f8ddd None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:31.725090 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8722dc7-f448-4744-8613-b4b33e9f8ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:31.725361 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8722dc7-f448-4744-8613-b4b33e9f8ddd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:31.726107 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8722dc7-f448-4744-8613-b4b33e9f8ddd None admin] RBAC: Adding query filter params (role.id=532b37d9076c45618cf352aafb85f35e, scope.domain.id=default, user.id=7ecff1fbd79248b7bcd1e69035be3515) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:18:31.726414 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8722dc7-f448-4744-8613-b4b33e9f8ddd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:31.727863 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8722dc7-f448-4744-8613-b4b33e9f8ddd None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:31.735875 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 11/20] 10.222.0.49 () {56 vars in 1322 bytes} [Mon Sep 26 10:18:31 2022] GET /identity/v3/role_assignments?user.id=7ecff1fbd79248b7bcd1e69035be3515&scope.domain.id=default&role.id=532b37d9076c45618cf352aafb85f35e => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:18:31.746205 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ffe7b0-a104-49aa-a8d9-49945987ff25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:31.746840 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffe7b0-a104-49aa-a8d9-49945987ff25 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:31.747123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffe7b0-a104-49aa-a8d9-49945987ff25 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:31.747409 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffe7b0-a104-49aa-a8d9-49945987ff25 None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:31.748078 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ffe7b0-a104-49aa-a8d9-49945987ff25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:18:31.748384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ffe7b0-a104-49aa-a8d9-49945987ff25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:31.749821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ffe7b0-a104-49aa-a8d9-49945987ff25 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:31.769782 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 10/21] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:18:34.151714 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abd28c-5653-4b1a-9854-7b7fe2e3e953 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:34.152223 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abd28c-5653-4b1a-9854-7b7fe2e3e953 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:34.152223 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abd28c-5653-4b1a-9854-7b7fe2e3e953 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:34.153510 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 12/22] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:18:34.167429 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93e2fe-2c97-442b-a856-9e99e5cc2cdc None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:34.167906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93e2fe-2c97-442b-a856-9e99e5cc2cdc None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:34.168400 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93e2fe-2c97-442b-a856-9e99e5cc2cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:34.195452 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-4d93e2fe-2c97-442b-a856-9e99e5cc2cdc None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:18:34.196674 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-4d93e2fe-2c97-442b-a856-9e99e5cc2cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:18:34.288254 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 11/23] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:18:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:18:34.741747 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1df58-801c-434d-896b-084dafeba3d7 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:34.742168 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1df58-801c-434d-896b-084dafeba3d7 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:34.742360 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1df58-801c-434d-896b-084dafeba3d7 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:34.743710 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 13/24] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:18:34.749779 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf3920-6b6f-4523-b128-e50c5e86040f None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:34.750107 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf3920-6b6f-4523-b128-e50c5e86040f None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:34.750391 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf3920-6b6f-4523-b128-e50c5e86040f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:34.775188 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-6cdf3920-6b6f-4523-b128-e50c5e86040f None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:18:34.776430 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-6cdf3920-6b6f-4523-b128-e50c5e86040f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:18:34.804838 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:34.858033 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1df58-801c-434d-896b-084dafeba3d7 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:34.880464 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 12/25] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:18:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:18:35.001840 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fafb5d-45ef-4506-b8dc-413b4049cee2 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:35.002161 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fafb5d-45ef-4506-b8dc-413b4049cee2 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:35.002343 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fafb5d-45ef-4506-b8dc-413b4049cee2 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:35.003963 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 13/26] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:18:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:18:35.012198 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd01de-b3a0-4232-8383-d985211cfe1c None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:35.012512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd01de-b3a0-4232-8383-d985211cfe1c None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:35.012776 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd01de-b3a0-4232-8383-d985211cfe1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:35.023917 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1df58-801c-434d-896b-084dafeba3d7 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:35.050856 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-f1dd01de-b3a0-4232-8383-d985211cfe1c None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:18:35.052067 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-f1dd01de-b3a0-4232-8383-d985211cfe1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:18:35.094254 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed62f51-a390-4ebc-9efe-5c08c923bd49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:35.094887 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed62f51-a390-4ebc-9efe-5c08c923bd49 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:35.095165 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed62f51-a390-4ebc-9efe-5c08c923bd49 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:35.095434 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed62f51-a390-4ebc-9efe-5c08c923bd49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:35.117298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed62f51-a390-4ebc-9efe-5c08c923bd49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:18:35.138945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-5ed62f51-a390-4ebc-9efe-5c08c923bd49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:18:35.138945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:18:35.138945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:18:35.138945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:18:35.138945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:18:35.138945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:18:35.138945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:18:35.138945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:18:35.138945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:18:35.138945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:18:35.138945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:18:35.138945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:18:35.138945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:18:35.138945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:18:35.138945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:18:35.138945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:18:35.139832 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:18:35.139832 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:18:35.139832 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:18:35.139832 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:18:35.139832 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:18:35.139832 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:18:35.139832 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:18:35.139832 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:18:35.139832 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:18:35.139832 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:18:35.139832 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:18:35.139832 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:18:35.139832 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:18:35.139832 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:18:35.139832 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:18:35.139832 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:18:35.140606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:18:35.140606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:18:35.140606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:18:35.140606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:18:35.140606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:18:35.140606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:18:35.140606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:18:35.140606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:18:35.140606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:18:35.140606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:18:35.140606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:18:35.140606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:18:35.140606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 14/27] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:18:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:18:35.141751 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab1511-4e7e-4dbd-a9cc-80a9b9e42991 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:35.141751 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab1511-4e7e-4dbd-a9cc-80a9b9e42991 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:35.141905 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab1511-4e7e-4dbd-a9cc-80a9b9e42991 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:35.150221 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 15/28] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:18:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:18:35.151255 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab1511-4e7e-4dbd-a9cc-80a9b9e42991 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:35.196514 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:35.197542 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 14/29] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:18:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 10:18:35.200379 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6363e-5140-4112-989f-2ac9089207c2 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:35.201220 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6363e-5140-4112-989f-2ac9089207c2 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:35.201500 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6363e-5140-4112-989f-2ac9089207c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:35.241565 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-bac6363e-5140-4112-989f-2ac9089207c2 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:18:35.243971 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-bac6363e-5140-4112-989f-2ac9089207c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:18:35.345455 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab1511-4e7e-4dbd-a9cc-80a9b9e42991 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:35.373543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 15/30] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:18:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:18:35.374201 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6363e-5140-4112-989f-2ac9089207c2 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:35.408658 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:35.419119 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb60883-fea0-45a7-9719-ece8fd8705bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:35.420281 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb60883-fea0-45a7-9719-ece8fd8705bf None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:35.420555 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb60883-fea0-45a7-9719-ece8fd8705bf None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:35.420821 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb60883-fea0-45a7-9719-ece8fd8705bf None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:35.421486 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb60883-fea0-45a7-9719-ece8fd8705bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:18:35.421787 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb60883-fea0-45a7-9719-ece8fd8705bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:35.423299 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb60883-fea0-45a7-9719-ece8fd8705bf None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:35.458159 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 16/31] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:18:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:35.464049 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb60883-fea0-45a7-9719-ece8fd8705bf None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:35.587997 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6363e-5140-4112-989f-2ac9089207c2 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:35.625410 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d4e14-1b46-44f0-a16d-432ac6c46e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:35.625966 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d4e14-1b46-44f0-a16d-432ac6c46e6a None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:35.626270 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d4e14-1b46-44f0-a16d-432ac6c46e6a None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:35.626548 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d4e14-1b46-44f0-a16d-432ac6c46e6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:35.630312 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:35.638599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d4e14-1b46-44f0-a16d-432ac6c46e6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:35.640860 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d4e14-1b46-44f0-a16d-432ac6c46e6a None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:35.656926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-d21d4e14-1b46-44f0-a16d-432ac6c46e6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:18:35.656926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:18:35.656926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:18:35.656926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:18:35.656926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:18:35.656926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:18:35.656926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:18:35.656926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:18:35.656926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:18:35.656926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:18:35.656926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:18:35.656926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:18:35.656926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:18:35.656926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:18:35.656926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:18:35.656926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:18:35.659340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:18:35.659340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:18:35.659340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:18:35.659340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:18:35.659340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:18:35.659340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:18:35.659340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:18:35.659340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:18:35.659340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:18:35.659340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:18:35.659340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:18:35.659340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:18:35.659340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:18:35.659340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:18:35.659340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:18:35.659340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:18:35.659340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:18:35.661441 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:18:35.661441 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:18:35.661441 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:18:35.661790 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb60883-fea0-45a7-9719-ece8fd8705bf None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:35.661925 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 16/32] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:18:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:18:35.661925 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d4e14-1b46-44f0-a16d-432ac6c46e6a None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:35.705067 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ccd0e0-6958-4925-90b7-9f07a009746c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:35.706054 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ccd0e0-6958-4925-90b7-9f07a009746c None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:35.706557 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ccd0e0-6958-4925-90b7-9f07a009746c None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:35.707029 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ccd0e0-6958-4925-90b7-9f07a009746c None admin] PATH_INFO: `/v3/users/7ecff1fbd79248b7bcd1e69035be3515` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:35.708367 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ccd0e0-6958-4925-90b7-9f07a009746c None admin] RBAC: Authorizing `identity:get_user(user_id=7ecff1fbd79248b7bcd1e69035be3515)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:35.710014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ccd0e0-6958-4925-90b7-9f07a009746c None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:35.711815 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 17/33] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:18:35 2022] GET /identity/v3/users/7ecff1fbd79248b7bcd1e69035be3515 => generated 247 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:35.712778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ccd0e0-6958-4925-90b7-9f07a009746c None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:35.737994 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:35.775938 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d4e14-1b46-44f0-a16d-432ac6c46e6a None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:35.821225 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c9217-0e86-4ca8-b946-c4e72d367d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:35.821936 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c9217-0e86-4ca8-b946-c4e72d367d46 None admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:35.822210 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c9217-0e86-4ca8-b946-c4e72d367d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:35.822429 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c9217-0e86-4ca8-b946-c4e72d367d46 None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:35.823113 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2c9217-0e86-4ca8-b946-c4e72d367d46 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:35.824516 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2c9217-0e86-4ca8-b946-c4e72d367d46 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:35.830840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ccd0e0-6958-4925-90b7-9f07a009746c None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:35.870549 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b82f87-b4df-42bd-b9b8-c77884a5791c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:35.871162 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b82f87-b4df-42bd-b9b8-c77884a5791c None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:35.871444 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b82f87-b4df-42bd-b9b8-c77884a5791c None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:35.871712 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b82f87-b4df-42bd-b9b8-c77884a5791c None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:35.872351 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b82f87-b4df-42bd-b9b8-c77884a5791c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:18:35.872677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b82f87-b4df-42bd-b9b8-c77884a5791c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:35.874058 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b82f87-b4df-42bd-b9b8-c77884a5791c None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:35.895017 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 18/34] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:18:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:35.895746 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b82f87-b4df-42bd-b9b8-c77884a5791c None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:35.914300 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:35.955392 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 17/35] 10.222.0.49 () {60 vars in 1131 bytes} [Mon Sep 26 10:18:35 2022] POST /identity/v3/roles => generated 225 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:18:35.956378 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c9217-0e86-4ca8-b946-c4e72d367d46 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:36.005729 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b82f87-b4df-42bd-b9b8-c77884a5791c None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:36.043911 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964209d2-2787-4c32-96d1-b4258161cde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:36.045103 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964209d2-2787-4c32-96d1-b4258161cde8 None admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:36.045388 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964209d2-2787-4c32-96d1-b4258161cde8 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:36.045671 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964209d2-2787-4c32-96d1-b4258161cde8 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:36.046403 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964209d2-2787-4c32-96d1-b4258161cde8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:36.047835 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964209d2-2787-4c32-96d1-b4258161cde8 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:36.072529 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c9217-0e86-4ca8-b946-c4e72d367d46 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:36.103094 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:36.115447 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40f2d58-b41c-42f6-a349-162f30d015bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:36.116283 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f2d58-b41c-42f6-a349-162f30d015bb None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:36.120020 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f2d58-b41c-42f6-a349-162f30d015bb None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:36.120331 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f2d58-b41c-42f6-a349-162f30d015bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:36.121710 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40f2d58-b41c-42f6-a349-162f30d015bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:36.123579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40f2d58-b41c-42f6-a349-162f30d015bb None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:36.126347 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 18/36] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:18:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:36.127612 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e657928-5e4c-4e79-ad5d-875f9d3c47a7 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:36.127899 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e657928-5e4c-4e79-ad5d-875f9d3c47a7 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:36.128179 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e657928-5e4c-4e79-ad5d-875f9d3c47a7 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:36.129393 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 19/37] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:18:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:18:36.130169 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e657928-5e4c-4e79-ad5d-875f9d3c47a7 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:36.181219 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 19/38] 10.222.0.49 () {60 vars in 1131 bytes} [Mon Sep 26 10:18:35 2022] POST /identity/v3/roles => generated 219 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:18:36.182802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5eb0a8-abb9-40bd-8a15-eda63ed19ccc None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:36.183078 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5eb0a8-abb9-40bd-8a15-eda63ed19ccc None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:36.183763 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5eb0a8-abb9-40bd-8a15-eda63ed19ccc None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:36.184994 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 20/39] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:18:36.185726 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5eb0a8-abb9-40bd-8a15-eda63ed19ccc None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:36.282608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:36.311917 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5eb0a8-abb9-40bd-8a15-eda63ed19ccc None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:36.351214 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b939bd51-29f0-4de7-b9d7-4e9b776ddeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:36.351904 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939bd51-29f0-4de7-b9d7-4e9b776ddeb8 None admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:36.352187 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939bd51-29f0-4de7-b9d7-4e9b776ddeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:36.352470 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939bd51-29f0-4de7-b9d7-4e9b776ddeb8 None admin] PATH_INFO: `/v3/domains/default/users/7ecff1fbd79248b7bcd1e69035be3515/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:36.355661 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b939bd51-29f0-4de7-b9d7-4e9b776ddeb8 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=7ecff1fbd79248b7bcd1e69035be3515, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:36.357398 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b939bd51-29f0-4de7-b9d7-4e9b776ddeb8 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:36.374242 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 21/40] 10.222.0.49 () {58 vars in 1361 bytes} [Mon Sep 26 10:18:36 2022] PUT /identity/v3/domains/default/users/7ecff1fbd79248b7bcd1e69035be3515/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:18:36.378814 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e0558-21a0-4f8b-be73-830f1c002682 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:36.379097 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e0558-21a0-4f8b-be73-830f1c002682 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:36.379641 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e0558-21a0-4f8b-be73-830f1c002682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:36.382481 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e657928-5e4c-4e79-ad5d-875f9d3c47a7 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:36.402760 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-569e0558-21a0-4f8b-be73-830f1c002682 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:18:36.404506 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-569e0558-21a0-4f8b-be73-830f1c002682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:18:36.434163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735baf24-5941-4ed6-b99c-54321acee9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:36.434702 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735baf24-5941-4ed6-b99c-54321acee9a8 None admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:36.435505 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735baf24-5941-4ed6-b99c-54321acee9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:36.435782 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735baf24-5941-4ed6-b99c-54321acee9a8 None admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:36.436467 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735baf24-5941-4ed6-b99c-54321acee9a8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:36.437966 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735baf24-5941-4ed6-b99c-54321acee9a8 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:36.485942 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 20/41] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Sep 26 10:18:36 2022] POST /identity/v3/projects => generated 301 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:18:36.488467 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2023d1-e10e-4fca-9441-a99247b45229 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:36.488938 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2023d1-e10e-4fca-9441-a99247b45229 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:36.489467 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2023d1-e10e-4fca-9441-a99247b45229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:36.498582 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 22/42] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:18:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:18:36.513102 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-db2023d1-e10e-4fca-9441-a99247b45229 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:18:36.514823 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-db2023d1-e10e-4fca-9441-a99247b45229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:18:36.569553 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:36.601878 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 21/43] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:18:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:18:36.810310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e0558-21a0-4f8b-be73-830f1c002682 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:36.878354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:36.883887 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2023d1-e10e-4fca-9441-a99247b45229 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:36.922367 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e0558-21a0-4f8b-be73-830f1c002682 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:36.960854 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99a09a-e05f-4528-8923-e304b4eae0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:36.961645 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99a09a-e05f-4528-8923-e304b4eae0c5 None admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:36.961934 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99a09a-e05f-4528-8923-e304b4eae0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:36.962325 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99a09a-e05f-4528-8923-e304b4eae0c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:36.963599 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99a09a-e05f-4528-8923-e304b4eae0c5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:36.965157 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99a09a-e05f-4528-8923-e304b4eae0c5 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:36.981024 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 23/44] 10.222.0.49 () {60 vars in 1131 bytes} [Mon Sep 26 10:18:36 2022] POST /identity/v3/roles => generated 223 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:18:37.002473 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2023d1-e10e-4fca-9441-a99247b45229 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:37.045735 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ca065-c4bf-4283-8a2c-d74e4a4da35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:37.046636 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ca065-c4bf-4283-8a2c-d74e4a4da35d None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:37.047021 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ca065-c4bf-4283-8a2c-d74e4a4da35d None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:37.047378 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ca065-c4bf-4283-8a2c-d74e4a4da35d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:37.048727 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ca065-c4bf-4283-8a2c-d74e4a4da35d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:37.050345 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ca065-c4bf-4283-8a2c-d74e4a4da35d None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:37.052258 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 22/45] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:18:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:18:37.058097 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99a09a-e05f-4528-8923-e304b4eae0c5 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:37.171436 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99a09a-e05f-4528-8923-e304b4eae0c5 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:37.197757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:37.211133 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04397d97-526d-4778-949c-061f4a72c500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:37.211853 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04397d97-526d-4778-949c-061f4a72c500 None admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:37.212220 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04397d97-526d-4778-949c-061f4a72c500 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:37.212730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04397d97-526d-4778-949c-061f4a72c500 None admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:37.213785 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04397d97-526d-4778-949c-061f4a72c500 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:37.215352 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04397d97-526d-4778-949c-061f4a72c500 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:37.250458 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 24/46] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Sep 26 10:18:37 2022] POST /identity/v3/projects => generated 312 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:18:38.668971 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abcb96-95d6-4add-b662-d02b195a157c None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:38.670059 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abcb96-95d6-4add-b662-d02b195a157c None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:38.670059 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abcb96-95d6-4add-b662-d02b195a157c None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:38.671434 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 23/47] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:18:38.676817 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0662d51-f968-4449-b906-e923a3e0774e None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:38.677209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0662d51-f968-4449-b906-e923a3e0774e None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:38.677503 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0662d51-f968-4449-b906-e923a3e0774e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:38.696499 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-d0662d51-f968-4449-b906-e923a3e0774e None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:18:38.697702 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-d0662d51-f968-4449-b906-e923a3e0774e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:18:38.866288 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 25/48] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:18:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:18:39.140345 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16abcb96-95d6-4add-b662-d02b195a157c None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:39.189473 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:39.244814 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16abcb96-95d6-4add-b662-d02b195a157c None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:39.281603 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84217661-a80c-4a8a-8f50-986fd8031627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:39.282288 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84217661-a80c-4a8a-8f50-986fd8031627 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:39.282571 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84217661-a80c-4a8a-8f50-986fd8031627 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:39.282844 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84217661-a80c-4a8a-8f50-986fd8031627 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:39.294065 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84217661-a80c-4a8a-8f50-986fd8031627 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:18:39.299093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-84217661-a80c-4a8a-8f50-986fd8031627 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:18:39.299093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:18:39.299093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:18:39.299093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:18:39.299093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:18:39.299093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:18:39.299093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:18:39.299093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:18:39.299093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:18:39.299093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:18:39.299093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:18:39.299093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:18:39.299093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:18:39.299093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:18:39.299093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:18:39.299093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:18:39.299934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:18:39.299934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:18:39.299934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:18:39.299934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:18:39.299934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:18:39.299934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:18:39.299934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:18:39.299934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:18:39.299934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:18:39.299934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:18:39.299934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:18:39.299934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:18:39.299934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:18:39.299934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:18:39.299934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:18:39.299934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:18:39.300702 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:18:39.300702 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:18:39.300702 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:18:39.300702 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:18:39.300702 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:18:39.300702 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:18:39.300702 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:18:39.300702 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:18:39.300702 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:18:39.300702 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:18:39.300702 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:18:39.300702 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:18:39.300702 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 24/49] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:18:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:18:39.307241 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0662d51-f968-4449-b906-e923a3e0774e None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:39.378585 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a61bd-4d0e-4bbf-866a-2a7df3bc7298 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:39.378867 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a61bd-4d0e-4bbf-866a-2a7df3bc7298 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:39.379143 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a61bd-4d0e-4bbf-866a-2a7df3bc7298 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:39.380482 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 25/50] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:18:39.385261 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e8089-ebab-4eca-ab5a-2c4ca2340edc None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:39.385542 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e8089-ebab-4eca-ab5a-2c4ca2340edc None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:39.385818 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e8089-ebab-4eca-ab5a-2c4ca2340edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:39.405761 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-d87e8089-ebab-4eca-ab5a-2c4ca2340edc None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:18:39.406554 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0662d51-f968-4449-b906-e923a3e0774e None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:39.406912 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-d87e8089-ebab-4eca-ab5a-2c4ca2340edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:18:39.416705 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:39.443571 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb3d01-c33c-4aa0-8e6d-c38246883e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:39.444200 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb3d01-c33c-4aa0-8e6d-c38246883e73 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:39.444482 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb3d01-c33c-4aa0-8e6d-c38246883e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:39.444759 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb3d01-c33c-4aa0-8e6d-c38246883e73 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:39.445428 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eb3d01-c33c-4aa0-8e6d-c38246883e73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:18:39.445732 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eb3d01-c33c-4aa0-8e6d-c38246883e73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:39.447187 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eb3d01-c33c-4aa0-8e6d-c38246883e73 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:39.456474 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 26/51] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:18:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:39.461876 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb3d01-c33c-4aa0-8e6d-c38246883e73 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:39.476000 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 26/52] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:18:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 10:18:39.557060 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb3d01-c33c-4aa0-8e6d-c38246883e73 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:39.590228 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1347e6f-e335-4066-9db5-fc16946e77b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:39.590841 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1347e6f-e335-4066-9db5-fc16946e77b4 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:39.591120 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1347e6f-e335-4066-9db5-fc16946e77b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:39.591419 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1347e6f-e335-4066-9db5-fc16946e77b4 None admin] PATH_INFO: `/v3/users/7ecff1fbd79248b7bcd1e69035be3515` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:39.592547 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1347e6f-e335-4066-9db5-fc16946e77b4 None admin] RBAC: Authorizing `identity:get_user(user_id=7ecff1fbd79248b7bcd1e69035be3515)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:39.593975 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1347e6f-e335-4066-9db5-fc16946e77b4 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:39.595596 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 27/53] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:18:39 2022] GET /identity/v3/users/7ecff1fbd79248b7bcd1e69035be3515 => generated 247 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:18:39.601075 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e8089-ebab-4eca-ab5a-2c4ca2340edc None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:39.634403 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:39.702076 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e8089-ebab-4eca-ab5a-2c4ca2340edc None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:39.734328 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8e97b-09fc-484d-aba7-72045ff377e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:39.734947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8e97b-09fc-484d-aba7-72045ff377e1 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:39.735224 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8e97b-09fc-484d-aba7-72045ff377e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:39.735494 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8e97b-09fc-484d-aba7-72045ff377e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:39.736585 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce8e97b-09fc-484d-aba7-72045ff377e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:39.738036 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce8e97b-09fc-484d-aba7-72045ff377e1 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:39.739598 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 27/54] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:18:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:18:39.745104 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8e97b-09fc-484d-aba7-72045ff377e1 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:39.745217 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1347e6f-e335-4066-9db5-fc16946e77b4 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:39.786267 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:39.847130 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8e97b-09fc-484d-aba7-72045ff377e1 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:39.848745 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:39.851125 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1347e6f-e335-4066-9db5-fc16946e77b4 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:39.882754 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b034c103-9d6e-45df-8829-f8e13c794343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:39.883358 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034c103-9d6e-45df-8829-f8e13c794343 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:39.883634 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034c103-9d6e-45df-8829-f8e13c794343 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:39.883906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034c103-9d6e-45df-8829-f8e13c794343 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:39.884621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b034c103-9d6e-45df-8829-f8e13c794343 None admin] RBAC: Adding query filter params (role.id=532b37d9076c45618cf352aafb85f35e, scope.domain.id=default, user.id=7ecff1fbd79248b7bcd1e69035be3515) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:18:39.884922 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b034c103-9d6e-45df-8829-f8e13c794343 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:39.886158 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4f55f-ad8b-4c6c-ad71-1a9b795e2bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:39.886372 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b034c103-9d6e-45df-8829-f8e13c794343 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:39.886785 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4f55f-ad8b-4c6c-ad71-1a9b795e2bc1 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:39.887064 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4f55f-ad8b-4c6c-ad71-1a9b795e2bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:39.887343 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4f55f-ad8b-4c6c-ad71-1a9b795e2bc1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:39.888620 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d4f55f-ad8b-4c6c-ad71-1a9b795e2bc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:39.890084 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d4f55f-ad8b-4c6c-ad71-1a9b795e2bc1 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:39.892087 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 28/55] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:18:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:39.896101 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 28/56] 10.222.0.49 () {56 vars in 1322 bytes} [Mon Sep 26 10:18:39 2022] GET /identity/v3/role_assignments?user.id=7ecff1fbd79248b7bcd1e69035be3515&scope.domain.id=default&role.id=532b37d9076c45618cf352aafb85f35e => generated 530 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:39.897601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4f55f-ad8b-4c6c-ad71-1a9b795e2bc1 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:40.003409 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4f55f-ad8b-4c6c-ad71-1a9b795e2bc1 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:40.011374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:40.040507 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55ee95-1f82-44b5-9f0f-bca85ded0e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:40.041132 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55ee95-1f82-44b5-9f0f-bca85ded0e61 None admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:40.041414 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55ee95-1f82-44b5-9f0f-bca85ded0e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:40.041687 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55ee95-1f82-44b5-9f0f-bca85ded0e61 None admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:40.042468 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55ee95-1f82-44b5-9f0f-bca85ded0e61 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:40.043925 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55ee95-1f82-44b5-9f0f-bca85ded0e61 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:40.072135 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 29/57] 10.222.0.49 () {60 vars in 1140 bytes} [Mon Sep 26 10:18:39 2022] POST /identity/v3/projects => generated 298 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:18:42.218691 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7051dc6-fbda-42e9-ac74-f3320baeea24 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:42.219176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7051dc6-fbda-42e9-ac74-f3320baeea24 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:42.219176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7051dc6-fbda-42e9-ac74-f3320baeea24 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:42.220497 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 29/58] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:18:42.225710 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097389fb-8260-438c-8c75-62c7cfe4ec22 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:42.226160 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097389fb-8260-438c-8c75-62c7cfe4ec22 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:42.226284 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097389fb-8260-438c-8c75-62c7cfe4ec22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:42.244040 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-097389fb-8260-438c-8c75-62c7cfe4ec22 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:18:42.245212 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-097389fb-8260-438c-8c75-62c7cfe4ec22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:18:42.309273 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 30/59] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:18:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:18:42.573734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7051dc6-fbda-42e9-ac74-f3320baeea24 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:42.671621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7051dc6-fbda-42e9-ac74-f3320baeea24 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:42.704247 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbea8d4-5ea1-46e0-a4ae-3631cbb23a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:42.704866 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbea8d4-5ea1-46e0-a4ae-3631cbb23a48 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:42.705143 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbea8d4-5ea1-46e0-a4ae-3631cbb23a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:42.705415 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbea8d4-5ea1-46e0-a4ae-3631cbb23a48 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:42.706544 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbea8d4-5ea1-46e0-a4ae-3631cbb23a48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:42.707981 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbea8d4-5ea1-46e0-a4ae-3631cbb23a48 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:42.709516 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 30/60] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:18:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:42.714928 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097389fb-8260-438c-8c75-62c7cfe4ec22 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:42.798571 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:42.812222 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097389fb-8260-438c-8c75-62c7cfe4ec22 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:42.845512 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b5fc7f-1a28-43b6-8404-c60720a6c9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:42.846083 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5fc7f-1a28-43b6-8404-c60720a6c9de None admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:42.846368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5fc7f-1a28-43b6-8404-c60720a6c9de None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:42.846644 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5fc7f-1a28-43b6-8404-c60720a6c9de None admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:42.847301 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b5fc7f-1a28-43b6-8404-c60720a6c9de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:42.848740 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b5fc7f-1a28-43b6-8404-c60720a6c9de None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:42.875426 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 31/61] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 10:18:42 2022] POST /identity/v3/users => generated 275 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:18:46.172314 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4411dd6-b31f-4095-826b-3e5e2ed22438 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:46.173358 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4411dd6-b31f-4095-826b-3e5e2ed22438 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:46.173845 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4411dd6-b31f-4095-826b-3e5e2ed22438 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:46.175434 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 31/62] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:18:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:18:46.181230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f8f42-0324-422d-b106-2f75417d4931 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:46.182091 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f8f42-0324-422d-b106-2f75417d4931 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:46.182592 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f8f42-0324-422d-b106-2f75417d4931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:46.210276 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-a08f8f42-0324-422d-b106-2f75417d4931 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:18:46.211817 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-a08f8f42-0324-422d-b106-2f75417d4931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:18:46.311289 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 32/63] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:18:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:18:46.434507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d18bbc-cb22-436b-8a2e-529c75628c53 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:46.434507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d18bbc-cb22-436b-8a2e-529c75628c53 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:46.434507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d18bbc-cb22-436b-8a2e-529c75628c53 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:46.434507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 32/64] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:18:46.439899 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29970b86-1ae6-4232-b710-e47e4d71cef3 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:46.440186 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29970b86-1ae6-4232-b710-e47e4d71cef3 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:46.440466 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29970b86-1ae6-4232-b710-e47e4d71cef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:46.452506 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227172a7-68bd-4444-9818-376c565e4e3f None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:46.453101 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227172a7-68bd-4444-9818-376c565e4e3f None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:46.453602 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227172a7-68bd-4444-9818-376c565e4e3f None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:46.455432 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 33/65] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:18:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:18:46.463132 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-29970b86-1ae6-4232-b710-e47e4d71cef3 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:18:46.464745 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-29970b86-1ae6-4232-b710-e47e4d71cef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:18:46.466060 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307beb2-0bc1-4459-a262-18bae17d2da0 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:46.466912 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307beb2-0bc1-4459-a262-18bae17d2da0 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:46.467197 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307beb2-0bc1-4459-a262-18bae17d2da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:46.503172 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-f307beb2-0bc1-4459-a262-18bae17d2da0 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:18:46.504456 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-f307beb2-0bc1-4459-a262-18bae17d2da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:18:46.544120 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 33/66] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:18:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:18:46.562340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:46.633776 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42845bcf-7a17-4ebb-8862-c2db785fd65b None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:46.634428 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42845bcf-7a17-4ebb-8862-c2db785fd65b None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:46.634645 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 34/67] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:18:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:18:46.635094 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42845bcf-7a17-4ebb-8862-c2db785fd65b None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:46.636746 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 34/68] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:18:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:18:46.642862 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b15bf-db12-49f2-8c18-7fad0965b97b None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:46.643158 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b15bf-db12-49f2-8c18-7fad0965b97b None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:46.643443 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b15bf-db12-49f2-8c18-7fad0965b97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:46.671175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-fd3b15bf-db12-49f2-8c18-7fad0965b97b None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:18:46.672400 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-fd3b15bf-db12-49f2-8c18-7fad0965b97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:18:46.704322 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42845bcf-7a17-4ebb-8862-c2db785fd65b None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:46.711326 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:46.788081 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 35/69] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:18:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:18:46.854272 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d32be-f5be-4ecd-ae17-8bdc3f706c8b None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:46.854915 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d32be-f5be-4ecd-ae17-8bdc3f706c8b None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:46.855398 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d32be-f5be-4ecd-ae17-8bdc3f706c8b None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:46.857001 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 36/70] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:18:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:18:46.862618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834baa17-ba92-44bd-9778-0ac2768266ab None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:46.862899 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834baa17-ba92-44bd-9778-0ac2768266ab None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:46.863172 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834baa17-ba92-44bd-9778-0ac2768266ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:46.890576 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42845bcf-7a17-4ebb-8862-c2db785fd65b None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:46.891950 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-834baa17-ba92-44bd-9778-0ac2768266ab None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:18:46.893371 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-834baa17-ba92-44bd-9778-0ac2768266ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:18:46.924923 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385d285-6dbf-4399-8a93-9df82cde8635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:46.925801 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385d285-6dbf-4399-8a93-9df82cde8635 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:46.926251 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385d285-6dbf-4399-8a93-9df82cde8635 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:46.926676 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385d285-6dbf-4399-8a93-9df82cde8635 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:46.938261 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3385d285-6dbf-4399-8a93-9df82cde8635 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:18:46.943297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-3385d285-6dbf-4399-8a93-9df82cde8635 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:18:46.943297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:18:46.943297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:18:46.943297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:18:46.943297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:18:46.943297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:18:46.943297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:18:46.943297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:18:46.943297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:18:46.943297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:18:46.943297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:18:46.943297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:18:46.943297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:18:46.943297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:18:46.943297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:18:46.943297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:18:46.944177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:18:46.944177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:18:46.944177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:18:46.944177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:18:46.944177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:18:46.944177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:18:46.944177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:18:46.944177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:18:46.944177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:18:46.944177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:18:46.944177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:18:46.944177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:18:46.944177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:18:46.944177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:18:46.944177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:18:46.944177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:18:46.944936 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:18:46.944936 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:18:46.944936 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:18:46.944936 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:18:46.944936 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:18:46.944936 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:18:46.944936 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:18:46.944936 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:18:46.944936 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:18:46.944936 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:18:46.944936 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:18:46.944936 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:18:46.947100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 35/71] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:18:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:18:46.953209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385d285-6dbf-4399-8a93-9df82cde8635 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:46.971812 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 37/72] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:18:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:18:46.972984 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834baa17-ba92-44bd-9778-0ac2768266ab None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:46.998118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:47.032665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:47.076461 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385d285-6dbf-4399-8a93-9df82cde8635 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:47.108616 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834baa17-ba92-44bd-9778-0ac2768266ab None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:47.114317 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322aa046-2330-4587-ae3c-15957509519d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:47.114953 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322aa046-2330-4587-ae3c-15957509519d None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:47.115232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322aa046-2330-4587-ae3c-15957509519d None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:47.115503 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322aa046-2330-4587-ae3c-15957509519d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:47.126576 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322aa046-2330-4587-ae3c-15957509519d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:18:47.131379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-322aa046-2330-4587-ae3c-15957509519d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:18:47.131379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:18:47.131379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:18:47.131379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:18:47.131379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:18:47.131379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:18:47.131379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:18:47.131379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:18:47.131379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:18:47.131379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:18:47.131379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:18:47.131379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:18:47.131379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:18:47.131379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:18:47.131379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:18:47.131379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:18:47.132215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:18:47.132215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:18:47.132215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:18:47.132215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:18:47.132215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:18:47.132215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:18:47.132215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:18:47.132215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:18:47.132215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:18:47.132215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:18:47.132215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:18:47.132215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:18:47.132215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:18:47.132215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:18:47.132215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:18:47.132215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:18:47.132988 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:18:47.132988 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:18:47.132988 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:18:47.132988 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:18:47.132988 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:18:47.132988 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:18:47.132988 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:18:47.132988 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:18:47.132988 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:18:47.132988 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:18:47.132988 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:18:47.132988 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:18:47.132988 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 36/73] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:18:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:18:47.133685 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322aa046-2330-4587-ae3c-15957509519d None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:47.150845 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f688a08-77b7-4453-8515-5a629610614c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:47.151460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f688a08-77b7-4453-8515-5a629610614c None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:47.151744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f688a08-77b7-4453-8515-5a629610614c None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:47.152019 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f688a08-77b7-4453-8515-5a629610614c None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:47.152688 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f688a08-77b7-4453-8515-5a629610614c None admin] RBAC: Adding query filter params (name=member) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:18:47.152990 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f688a08-77b7-4453-8515-5a629610614c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:47.154441 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f688a08-77b7-4453-8515-5a629610614c None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:47.166779 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 38/74] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 10:18:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:47.167474 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f688a08-77b7-4453-8515-5a629610614c None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:47.251734 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322aa046-2330-4587-ae3c-15957509519d None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:47.256818 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:47.272224 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f688a08-77b7-4453-8515-5a629610614c None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:47.275967 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:47.287500 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2250f4f-c14f-4597-964d-ecdf59e382ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:47.288134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2250f4f-c14f-4597-964d-ecdf59e382ec None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:47.288420 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2250f4f-c14f-4597-964d-ecdf59e382ec None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:47.288698 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2250f4f-c14f-4597-964d-ecdf59e382ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:47.299486 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2250f4f-c14f-4597-964d-ecdf59e382ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:18:47.304403 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-e2250f4f-c14f-4597-964d-ecdf59e382ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 10:18:47.304403 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:18:47.304403 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:18:47.304403 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:18:47.304403 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:18:47.304403 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:18:47.304403 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:18:47.304403 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:18:47.304403 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:18:47.304403 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:18:47.304403 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:18:47.304403 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:18:47.304403 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:18:47.304403 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:18:47.304403 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:18:47.304403 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:18:47.305312 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:18:47.305312 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:18:47.305312 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:18:47.305312 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:18:47.305312 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:18:47.305312 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:18:47.305312 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:18:47.305312 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:18:47.305312 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:18:47.305312 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:18:47.305312 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:18:47.305312 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:18:47.305312 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:18:47.305312 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:18:47.305312 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:18:47.305312 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:18:47.306373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:18:47.306373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:18:47.306373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:18:47.306373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:18:47.306373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:18:47.306373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:18:47.306373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:18:47.306373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:18:47.306373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:18:47.306373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:18:47.306373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 10:18:47.306373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:18:47.306373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 37/75] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 10:18:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:18:47.307313 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2250f4f-c14f-4597-964d-ecdf59e382ec None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:47.311714 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c57b6-d422-4686-9f2b-60f4d8e0a3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:47.312363 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c57b6-d422-4686-9f2b-60f4d8e0a3e2 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:47.312642 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c57b6-d422-4686-9f2b-60f4d8e0a3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:47.312911 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c57b6-d422-4686-9f2b-60f4d8e0a3e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:47.323923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c57b6-d422-4686-9f2b-60f4d8e0a3e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:18:47.329075 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-840c57b6-d422-4686-9f2b-60f4d8e0a3e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:18:47.329075 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:18:47.329075 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:18:47.329075 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:18:47.329075 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:18:47.329075 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:18:47.329075 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:18:47.329075 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:18:47.329075 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:18:47.329075 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:18:47.329075 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:18:47.329075 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:18:47.329075 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:18:47.329075 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:18:47.329075 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:18:47.329075 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:18:47.330012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:18:47.330012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:18:47.330012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:18:47.330012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:18:47.330012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:18:47.330012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:18:47.330012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:18:47.330012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:18:47.330012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:18:47.330012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:18:47.330012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:18:47.330012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:18:47.330012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:18:47.330012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:18:47.330012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:18:47.330012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:18:47.332338 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:18:47.332338 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:18:47.332338 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:18:47.332338 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:18:47.332338 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:18:47.332338 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:18:47.332338 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:18:47.332338 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:18:47.332338 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:18:47.332338 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:18:47.332338 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:18:47.332338 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:18:47.332338 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 39/76] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:18:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:18:47.332338 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c57b6-d422-4686-9f2b-60f4d8e0a3e2 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:47.409174 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2250f4f-c14f-4597-964d-ecdf59e382ec None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:47.429668 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c57b6-d422-4686-9f2b-60f4d8e0a3e2 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:47.443107 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955e5db-cf42-463b-a36e-c910adc3544a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:47.443724 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955e5db-cf42-463b-a36e-c910adc3544a None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:47.447394 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955e5db-cf42-463b-a36e-c910adc3544a None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:47.447695 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955e5db-cf42-463b-a36e-c910adc3544a None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:47.448375 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955e5db-cf42-463b-a36e-c910adc3544a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:18:47.448688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955e5db-cf42-463b-a36e-c910adc3544a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:47.450117 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955e5db-cf42-463b-a36e-c910adc3544a None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:47.459449 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 38/77] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:18:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:47.460189 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955e5db-cf42-463b-a36e-c910adc3544a None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:47.464434 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268bf05-e12c-46e8-92f0-14ce363db20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:47.465073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268bf05-e12c-46e8-92f0-14ce363db20a None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:47.465357 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268bf05-e12c-46e8-92f0-14ce363db20a None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:47.465631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268bf05-e12c-46e8-92f0-14ce363db20a None admin] PATH_INFO: `/v3/users/4cf56ce5f5694a0499199ccd4ee6ab75` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:47.479416 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268bf05-e12c-46e8-92f0-14ce363db20a None admin] RBAC: Authorizing `identity:get_user(user_id=4cf56ce5f5694a0499199ccd4ee6ab75)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:47.480885 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268bf05-e12c-46e8-92f0-14ce363db20a None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:47.483038 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 40/78] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:18:47 2022] GET /identity/v3/users/4cf56ce5f5694a0499199ccd4ee6ab75 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:47.483902 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268bf05-e12c-46e8-92f0-14ce363db20a None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:47.489603 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:47.507881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:47.560777 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955e5db-cf42-463b-a36e-c910adc3544a None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:47.583484 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268bf05-e12c-46e8-92f0-14ce363db20a None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:47.602628 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8dc25-9ce3-4a59-bc3c-1081dc2d9799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:47.603422 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8dc25-9ce3-4a59-bc3c-1081dc2d9799 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:47.603756 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8dc25-9ce3-4a59-bc3c-1081dc2d9799 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:47.604073 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8dc25-9ce3-4a59-bc3c-1081dc2d9799 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:47.605216 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df8dc25-9ce3-4a59-bc3c-1081dc2d9799 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:47.606704 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df8dc25-9ce3-4a59-bc3c-1081dc2d9799 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:47.608826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 39/79] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:18:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:47.610865 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8dc25-9ce3-4a59-bc3c-1081dc2d9799 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:47.631255 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965ed40-5b8d-4716-b6a8-f3f345f58827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:47.631862 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965ed40-5b8d-4716-b6a8-f3f345f58827 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:47.632141 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965ed40-5b8d-4716-b6a8-f3f345f58827 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:47.632412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965ed40-5b8d-4716-b6a8-f3f345f58827 None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:47.633075 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965ed40-5b8d-4716-b6a8-f3f345f58827 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:18:47.633378 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965ed40-5b8d-4716-b6a8-f3f345f58827 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:47.634788 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965ed40-5b8d-4716-b6a8-f3f345f58827 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:47.642806 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 41/80] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:18:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:47.643605 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965ed40-5b8d-4716-b6a8-f3f345f58827 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:47.708125 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:47.709433 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8dc25-9ce3-4a59-bc3c-1081dc2d9799 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:47.723530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:47.745359 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965ed40-5b8d-4716-b6a8-f3f345f58827 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:47.745569 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c258c06-b88d-4beb-a142-6a7f2fd52fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:47.746219 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c258c06-b88d-4beb-a142-6a7f2fd52fff None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:47.746503 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c258c06-b88d-4beb-a142-6a7f2fd52fff None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:47.746780 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c258c06-b88d-4beb-a142-6a7f2fd52fff None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:47.747495 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c258c06-b88d-4beb-a142-6a7f2fd52fff None admin] RBAC: Adding query filter params (name=member) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:18:47.747983 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c258c06-b88d-4beb-a142-6a7f2fd52fff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:47.749366 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c258c06-b88d-4beb-a142-6a7f2fd52fff None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:47.757476 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 40/81] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 10:18:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:47.758152 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c258c06-b88d-4beb-a142-6a7f2fd52fff None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:47.778348 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35736e-d2f8-403e-99c8-348188fd8e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:47.779483 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35736e-d2f8-403e-99c8-348188fd8e40 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:47.779764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35736e-d2f8-403e-99c8-348188fd8e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:47.780082 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35736e-d2f8-403e-99c8-348188fd8e40 None admin] PATH_INFO: `/v3/users/7ecff1fbd79248b7bcd1e69035be3515` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:47.781229 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f35736e-d2f8-403e-99c8-348188fd8e40 None admin] RBAC: Authorizing `identity:get_user(user_id=7ecff1fbd79248b7bcd1e69035be3515)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:47.782723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f35736e-d2f8-403e-99c8-348188fd8e40 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:47.784349 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 42/82] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:18:47 2022] GET /identity/v3/users/7ecff1fbd79248b7bcd1e69035be3515 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:47.785091 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35736e-d2f8-403e-99c8-348188fd8e40 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:47.865933 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c258c06-b88d-4beb-a142-6a7f2fd52fff None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:47.888390 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35736e-d2f8-403e-99c8-348188fd8e40 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:47.899473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c9582-c35a-43fa-b97b-fd690c4c83d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:47.900079 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c9582-c35a-43fa-b97b-fd690c4c83d1 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:47.900367 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c9582-c35a-43fa-b97b-fd690c4c83d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:47.900642 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c9582-c35a-43fa-b97b-fd690c4c83d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:47.924309 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089d1e2-fe6a-4215-a372-ab4e75bf539e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:47.924914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089d1e2-fe6a-4215-a372-ab4e75bf539e None admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:47.925201 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089d1e2-fe6a-4215-a372-ab4e75bf539e None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:47.925475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089d1e2-fe6a-4215-a372-ab4e75bf539e None admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:47.926165 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c089d1e2-fe6a-4215-a372-ab4e75bf539e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:47.927608 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c089d1e2-fe6a-4215-a372-ab4e75bf539e None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:47.937956 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:47.945294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:47.955877 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 43/83] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 10:18:47 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:18:47.956853 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089d1e2-fe6a-4215-a372-ab4e75bf539e None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:47.972949 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547c9582-c35a-43fa-b97b-fd690c4c83d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:47.974635 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547c9582-c35a-43fa-b97b-fd690c4c83d1 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:48.086153 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089d1e2-fe6a-4215-a372-ab4e75bf539e None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:48.087565 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 41/84] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:18:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 10:18:48.088429 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c9582-c35a-43fa-b97b-fd690c4c83d1 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:48.129214 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575fceac-1522-4d28-8926-e3e22ddf4b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:48.130217 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575fceac-1522-4d28-8926-e3e22ddf4b2b None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:48.130504 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575fceac-1522-4d28-8926-e3e22ddf4b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:48.131446 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575fceac-1522-4d28-8926-e3e22ddf4b2b None admin] PATH_INFO: `/v3/users/4cf56ce5f5694a0499199ccd4ee6ab75` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:48.131941 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575fceac-1522-4d28-8926-e3e22ddf4b2b None admin] RBAC: Authorizing `identity:get_user(user_id=4cf56ce5f5694a0499199ccd4ee6ab75)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:48.133676 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575fceac-1522-4d28-8926-e3e22ddf4b2b None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:48.135468 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 44/85] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:18:47 2022] GET /identity/v3/users/4cf56ce5f5694a0499199ccd4ee6ab75 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:48.135843 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575fceac-1522-4d28-8926-e3e22ddf4b2b None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:48.179565 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:48.208057 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:48.208640 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c9582-c35a-43fa-b97b-fd690c4c83d1 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:48.258354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fbc722-5df4-4bca-a11e-395157cad281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:48.259056 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fbc722-5df4-4bca-a11e-395157cad281 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:48.259595 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fbc722-5df4-4bca-a11e-395157cad281 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:48.259900 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fbc722-5df4-4bca-a11e-395157cad281 None admin] PATH_INFO: `/v3/users/4cf56ce5f5694a0499199ccd4ee6ab75` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:48.261106 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fbc722-5df4-4bca-a11e-395157cad281 None admin] RBAC: Authorizing `identity:get_user(user_id=4cf56ce5f5694a0499199ccd4ee6ab75)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:48.261207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575fceac-1522-4d28-8926-e3e22ddf4b2b None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:48.262829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fbc722-5df4-4bca-a11e-395157cad281 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:48.265373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 42/86] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:18:48 2022] GET /identity/v3/users/4cf56ce5f5694a0499199ccd4ee6ab75 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:48.266206 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fbc722-5df4-4bca-a11e-395157cad281 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:48.299792 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c984599-4f9e-44a2-9f0a-6276856cc2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:48.300425 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c984599-4f9e-44a2-9f0a-6276856cc2d0 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:48.300703 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c984599-4f9e-44a2-9f0a-6276856cc2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:48.301022 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c984599-4f9e-44a2-9f0a-6276856cc2d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:48.381287 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fbc722-5df4-4bca-a11e-395157cad281 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:48.388416 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c984599-4f9e-44a2-9f0a-6276856cc2d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:48.390655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c984599-4f9e-44a2-9f0a-6276856cc2d0 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:48.420414 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd5168-d0cc-4f34-b00a-c4fe7eb9ca00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:48.421202 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd5168-d0cc-4f34-b00a-c4fe7eb9ca00 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:48.421588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd5168-d0cc-4f34-b00a-c4fe7eb9ca00 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:48.421968 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd5168-d0cc-4f34-b00a-c4fe7eb9ca00 None admin] PATH_INFO: `/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:48.423346 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fd5168-d0cc-4f34-b00a-c4fe7eb9ca00 None admin] RBAC: Authorizing `identity:get_project(project_id=f81f2060d8dd4c8a85edacc1547c6b3c)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:48.425477 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fd5168-d0cc-4f34-b00a-c4fe7eb9ca00 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:48.427645 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 43/87] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:18:48 2022] GET /identity/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:48.428518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd5168-d0cc-4f34-b00a-c4fe7eb9ca00 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:48.445032 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:48.473020 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:48.527647 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 45/88] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:18:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 10:18:48.528551 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c984599-4f9e-44a2-9f0a-6276856cc2d0 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:48.558313 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd5168-d0cc-4f34-b00a-c4fe7eb9ca00 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:48.601144 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6e031-abe4-407a-9dd2-dfb4fc1b65a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:48.601794 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6e031-abe4-407a-9dd2-dfb4fc1b65a9 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:48.602118 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6e031-abe4-407a-9dd2-dfb4fc1b65a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:48.602398 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6e031-abe4-407a-9dd2-dfb4fc1b65a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:48.644163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c984599-4f9e-44a2-9f0a-6276856cc2d0 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:48.678068 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c6e031-abe4-407a-9dd2-dfb4fc1b65a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:48.680040 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c6e031-abe4-407a-9dd2-dfb4fc1b65a9 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:48.685548 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44ca23-48ca-4783-9eb7-4e2db22b25c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:48.686197 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44ca23-48ca-4783-9eb7-4e2db22b25c5 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:48.686483 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44ca23-48ca-4783-9eb7-4e2db22b25c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:48.686762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44ca23-48ca-4783-9eb7-4e2db22b25c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:48.695635 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:48.719103 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:48.764759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44ca23-48ca-4783-9eb7-4e2db22b25c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:48.766503 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44ca23-48ca-4783-9eb7-4e2db22b25c5 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:48.801530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 44/89] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:18:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 10:18:48.801530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6e031-abe4-407a-9dd2-dfb4fc1b65a9 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:48.896834 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 46/90] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:18:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 10:18:48.897607 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44ca23-48ca-4783-9eb7-4e2db22b25c5 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:48.927536 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6e031-abe4-407a-9dd2-dfb4fc1b65a9 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:48.942174 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:48.968118 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d30da09-4150-450f-a509-248991e16d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:48.969301 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30da09-4150-450f-a509-248991e16d03 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:48.969765 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30da09-4150-450f-a509-248991e16d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:48.970455 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30da09-4150-450f-a509-248991e16d03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:48.971786 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d30da09-4150-450f-a509-248991e16d03 None admin] RBAC: Adding query filter params (role.id=dc7cfc688d354df4bc870fe8621c48bd, scope.project.id=f81f2060d8dd4c8a85edacc1547c6b3c, user.id=4cf56ce5f5694a0499199ccd4ee6ab75) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:18:48.972295 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d30da09-4150-450f-a509-248991e16d03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:48.973947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d30da09-4150-450f-a509-248991e16d03 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:48.980893 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:48.994055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 45/91] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:18:48 2022] GET /identity/v3/role_assignments?user.id=4cf56ce5f5694a0499199ccd4ee6ab75&scope.project.id=f81f2060d8dd4c8a85edacc1547c6b3c&role.id=dc7cfc688d354df4bc870fe8621c48bd => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:48.994055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d30da09-4150-450f-a509-248991e16d03 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:49.077384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44ca23-48ca-4783-9eb7-4e2db22b25c5 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:49.129049 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33a7a4-7b1d-4f82-8505-222dc729621c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:49.129725 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33a7a4-7b1d-4f82-8505-222dc729621c None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:49.130003 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33a7a4-7b1d-4f82-8505-222dc729621c None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:49.130295 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33a7a4-7b1d-4f82-8505-222dc729621c None admin] PATH_INFO: `/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:49.131442 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af33a7a4-7b1d-4f82-8505-222dc729621c None admin] RBAC: Authorizing `identity:get_project(project_id=f81f2060d8dd4c8a85edacc1547c6b3c)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:49.132901 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af33a7a4-7b1d-4f82-8505-222dc729621c None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:49.134900 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 47/92] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:18:48 2022] GET /identity/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:49.135687 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33a7a4-7b1d-4f82-8505-222dc729621c None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:49.240818 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d30da09-4150-450f-a509-248991e16d03 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:49.274879 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33a7a4-7b1d-4f82-8505-222dc729621c None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:49.286534 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f33bf3-4498-4a2a-a0bd-57ce5961aada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:49.287957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f33bf3-4498-4a2a-a0bd-57ce5961aada None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:49.288441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f33bf3-4498-4a2a-a0bd-57ce5961aada None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:49.288924 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f33bf3-4498-4a2a-a0bd-57ce5961aada None admin] PATH_INFO: `/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:49.290420 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f33bf3-4498-4a2a-a0bd-57ce5961aada None admin] RBAC: Authorizing `identity:get_project(project_id=f81f2060d8dd4c8a85edacc1547c6b3c)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:49.292230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f33bf3-4498-4a2a-a0bd-57ce5961aada None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:49.294362 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 46/93] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:18:48 2022] GET /identity/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c => generated 298 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:49.302985 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f33bf3-4498-4a2a-a0bd-57ce5961aada None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:49.303737 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:49.312834 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e575c-ea37-47c6-8b7e-16f94bbe8279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:49.313533 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e575c-ea37-47c6-8b7e-16f94bbe8279 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:49.314108 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e575c-ea37-47c6-8b7e-16f94bbe8279 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:49.314729 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e575c-ea37-47c6-8b7e-16f94bbe8279 None admin] PATH_INFO: `/v3/projects/51fc4712d2454146b6103cf60db3fb7a` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:49.316842 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223e575c-ea37-47c6-8b7e-16f94bbe8279 None admin] RBAC: Authorizing `identity:get_project(project_id=51fc4712d2454146b6103cf60db3fb7a)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:49.318480 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223e575c-ea37-47c6-8b7e-16f94bbe8279 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:49.320762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 48/94] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:18:49 2022] GET /identity/v3/projects/51fc4712d2454146b6103cf60db3fb7a => generated 312 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:49.321429 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e575c-ea37-47c6-8b7e-16f94bbe8279 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:49.435002 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e575c-ea37-47c6-8b7e-16f94bbe8279 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:49.444046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:49.475731 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8cb63-94ff-4946-89f0-be4c1c49948e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:49.476351 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8cb63-94ff-4946-89f0-be4c1c49948e None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:49.476635 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8cb63-94ff-4946-89f0-be4c1c49948e None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:49.476917 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8cb63-94ff-4946-89f0-be4c1c49948e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:49.477653 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af8cb63-94ff-4946-89f0-be4c1c49948e None admin] RBAC: Adding query filter params (role.id=d1109db57b004dd9a4f84a634ff76205, scope.project.id=f81f2060d8dd4c8a85edacc1547c6b3c, user.id=4cf56ce5f5694a0499199ccd4ee6ab75) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:18:49.477964 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af8cb63-94ff-4946-89f0-be4c1c49948e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:49.479617 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af8cb63-94ff-4946-89f0-be4c1c49948e None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:49.486126 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 49/95] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:18:49 2022] GET /identity/v3/role_assignments?user.id=4cf56ce5f5694a0499199ccd4ee6ab75&scope.project.id=f81f2060d8dd4c8a85edacc1547c6b3c&role.id=d1109db57b004dd9a4f84a634ff76205 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:49.487419 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8cb63-94ff-4946-89f0-be4c1c49948e None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:49.558648 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:49.604149 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f33bf3-4498-4a2a-a0bd-57ce5961aada None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:49.605649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8cb63-94ff-4946-89f0-be4c1c49948e None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:49.641404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e29db-b050-4666-a1d0-8afd030ce677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:49.642037 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e29db-b050-4666-a1d0-8afd030ce677 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:49.642320 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e29db-b050-4666-a1d0-8afd030ce677 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:49.642595 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e29db-b050-4666-a1d0-8afd030ce677 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:49.643331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157e29db-b050-4666-a1d0-8afd030ce677 None admin] RBAC: Adding query filter params (role.id=532b37d9076c45618cf352aafb85f35e, scope.project.id=f81f2060d8dd4c8a85edacc1547c6b3c, user.id=7ecff1fbd79248b7bcd1e69035be3515) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:18:49.643640 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157e29db-b050-4666-a1d0-8afd030ce677 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:49.645113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157e29db-b050-4666-a1d0-8afd030ce677 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:49.651226 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 47/96] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:18:49 2022] GET /identity/v3/role_assignments?user.id=7ecff1fbd79248b7bcd1e69035be3515&scope.project.id=f81f2060d8dd4c8a85edacc1547c6b3c&role.id=532b37d9076c45618cf352aafb85f35e => generated 259 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:49.671002 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790c965-1c0c-4992-b617-b5ad7a6992d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:49.671773 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790c965-1c0c-4992-b617-b5ad7a6992d3 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:49.672060 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790c965-1c0c-4992-b617-b5ad7a6992d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:49.672336 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790c965-1c0c-4992-b617-b5ad7a6992d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:49.673106 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c790c965-1c0c-4992-b617-b5ad7a6992d3 None admin] RBAC: Adding query filter params (role.id=dc7cfc688d354df4bc870fe8621c48bd, scope.project.id=51fc4712d2454146b6103cf60db3fb7a, user.id=4cf56ce5f5694a0499199ccd4ee6ab75) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:18:49.673415 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c790c965-1c0c-4992-b617-b5ad7a6992d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:49.674967 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c790c965-1c0c-4992-b617-b5ad7a6992d3 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:49.681614 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 50/97] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:18:49 2022] GET /identity/v3/role_assignments?user.id=4cf56ce5f5694a0499199ccd4ee6ab75&scope.project.id=51fc4712d2454146b6103cf60db3fb7a&role.id=dc7cfc688d354df4bc870fe8621c48bd => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:50.570157 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc07bc1-f781-470e-911c-08645414fdd2 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:50.570663 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc07bc1-f781-470e-911c-08645414fdd2 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:50.570768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc07bc1-f781-470e-911c-08645414fdd2 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:50.572061 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 48/98] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:18:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:18:50.583874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383c9ef-51fd-4516-9ecd-543c244cf13d None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:50.584326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383c9ef-51fd-4516-9ecd-543c244cf13d None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:50.584398 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383c9ef-51fd-4516-9ecd-543c244cf13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:50.606889 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-6383c9ef-51fd-4516-9ecd-543c244cf13d None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:18:50.608100 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-6383c9ef-51fd-4516-9ecd-543c244cf13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:18:50.630073 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da082df5-4def-4f62-9756-997512921452 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:50.630367 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da082df5-4def-4f62-9756-997512921452 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:50.630608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da082df5-4def-4f62-9756-997512921452 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:50.632083 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 49/99] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:18:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:18:50.637348 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca227af-fbcb-4324-b7ad-6afed97f5bfc None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:50.637855 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca227af-fbcb-4324-b7ad-6afed97f5bfc None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:50.638914 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca227af-fbcb-4324-b7ad-6afed97f5bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:50.658557 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-3ca227af-fbcb-4324-b7ad-6afed97f5bfc None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:18:50.660095 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-3ca227af-fbcb-4324-b7ad-6afed97f5bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:18:50.712602 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 51/100] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:18:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:18:50.733826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 50/101] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:18:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 10:18:51.170351 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383c9ef-51fd-4516-9ecd-543c244cf13d None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:51.246929 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:51.316914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383c9ef-51fd-4516-9ecd-543c244cf13d None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:51.335339 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca227af-fbcb-4324-b7ad-6afed97f5bfc None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:51.365118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330eb41f-1e7b-4cc7-a698-977f004b8aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:51.365752 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330eb41f-1e7b-4cc7-a698-977f004b8aef None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:51.366049 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330eb41f-1e7b-4cc7-a698-977f004b8aef None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:51.366327 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330eb41f-1e7b-4cc7-a698-977f004b8aef None admin] PATH_INFO: `/v3/domains/default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:51.368305 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330eb41f-1e7b-4cc7-a698-977f004b8aef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:51.369863 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330eb41f-1e7b-4cc7-a698-977f004b8aef None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:51.371821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 52/102] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:18:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:51.377362 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330eb41f-1e7b-4cc7-a698-977f004b8aef None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:51.408698 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:51.516938 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca227af-fbcb-4324-b7ad-6afed97f5bfc None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:51.533923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330eb41f-1e7b-4cc7-a698-977f004b8aef None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:51.574672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:51.581739 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439fd07-a80c-4736-8fad-6d27bc5cccb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:51.582409 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439fd07-a80c-4736-8fad-6d27bc5cccb0 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:51.582690 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439fd07-a80c-4736-8fad-6d27bc5cccb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:51.582967 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439fd07-a80c-4736-8fad-6d27bc5cccb0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:51.603421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de1dff-7d60-4828-baa6-25b4300d33c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:51.604173 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de1dff-7d60-4828-baa6-25b4300d33c3 None admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:51.604335 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de1dff-7d60-4828-baa6-25b4300d33c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:51.604638 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de1dff-7d60-4828-baa6-25b4300d33c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:51.605256 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de1dff-7d60-4828-baa6-25b4300d33c3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:51.608101 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de1dff-7d60-4828-baa6-25b4300d33c3 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:51.613996 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6439fd07-a80c-4736-8fad-6d27bc5cccb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:18:51.619025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-6439fd07-a80c-4736-8fad-6d27bc5cccb0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 10:18:51.619025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:18:51.619025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:18:51.619025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:18:51.619025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:18:51.619025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:18:51.619025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:18:51.619025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:18:51.619025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:18:51.619025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:18:51.619025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:18:51.619025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:18:51.619025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:18:51.619025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:18:51.619025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:18:51.619025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:18:51.619870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:18:51.619870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:18:51.619870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:18:51.619870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:18:51.619870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:18:51.619870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:18:51.619870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:18:51.619870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:18:51.619870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:18:51.619870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:18:51.619870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:18:51.619870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:18:51.619870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:18:51.619870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:18:51.619870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:18:51.619870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:18:51.620644 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:18:51.620644 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:18:51.620644 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:18:51.620644 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:18:51.620644 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:18:51.620644 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:18:51.620644 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:18:51.620644 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:18:51.620644 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:18:51.620644 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:18:51.620644 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 10:18:51.620644 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:18:51.620644 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 51/103] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:18:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:18:51.640505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439fd07-a80c-4736-8fad-6d27bc5cccb0 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:51.654263 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 53/104] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Sep 26 10:18:51 2022] POST /identity/v3/projects => generated 302 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:18:51.799380 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:51.802895 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439fd07-a80c-4736-8fad-6d27bc5cccb0 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:51.866749 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfc964-b7e7-4361-aefa-48075c621485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:51.867377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfc964-b7e7-4361-aefa-48075c621485 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:51.867659 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfc964-b7e7-4361-aefa-48075c621485 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:51.867932 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfc964-b7e7-4361-aefa-48075c621485 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:51.868608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dfc964-b7e7-4361-aefa-48075c621485 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:18:51.868905 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dfc964-b7e7-4361-aefa-48075c621485 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:51.870317 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dfc964-b7e7-4361-aefa-48075c621485 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:51.884706 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 52/105] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 10:18:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:18:51.890305 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de1dff-7d60-4828-baa6-25b4300d33c3 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:52.002657 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de1dff-7d60-4828-baa6-25b4300d33c3 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:52.038323 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaddd2a-1cd3-4e0d-8dc0-f545d680b512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:52.039001 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaddd2a-1cd3-4e0d-8dc0-f545d680b512 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:52.039284 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaddd2a-1cd3-4e0d-8dc0-f545d680b512 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:52.039554 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaddd2a-1cd3-4e0d-8dc0-f545d680b512 None admin] PATH_INFO: `/v3/users/9b1ac2894c834148b68b48b22b199c2d` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:52.053937 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaddd2a-1cd3-4e0d-8dc0-f545d680b512 None admin] RBAC: Authorizing `identity:get_user(user_id=9b1ac2894c834148b68b48b22b199c2d)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:52.055375 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaddd2a-1cd3-4e0d-8dc0-f545d680b512 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:52.057213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 54/106] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:18:51 2022] GET /identity/v3/users/9b1ac2894c834148b68b48b22b199c2d => generated 289 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:52.063307 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfc964-b7e7-4361-aefa-48075c621485 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:52.188800 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfc964-b7e7-4361-aefa-48075c621485 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:52.231226 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6e89b-2503-4649-b68b-476369f57f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:52.232814 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6e89b-2503-4649-b68b-476369f57f88 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:52.232814 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6e89b-2503-4649-b68b-476369f57f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:52.233545 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6e89b-2503-4649-b68b-476369f57f88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:52.274142 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:52.324847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f6e89b-2503-4649-b68b-476369f57f88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:52.326296 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f6e89b-2503-4649-b68b-476369f57f88 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:52.465555 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 53/107] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:18:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 10:18:52.472120 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaddd2a-1cd3-4e0d-8dc0-f545d680b512 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:52.484118 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df36bda-f23f-4aa9-b15c-20a89c44a047 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:52.484401 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df36bda-f23f-4aa9-b15c-20a89c44a047 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:52.486208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df36bda-f23f-4aa9-b15c-20a89c44a047 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:52.486208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 54/108] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:18:52.494666 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f751a0e-3430-49e7-97da-58327ee57a5f None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:52.494946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f751a0e-3430-49e7-97da-58327ee57a5f None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:52.495219 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f751a0e-3430-49e7-97da-58327ee57a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:52.517381 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:52.531529 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-3f751a0e-3430-49e7-97da-58327ee57a5f None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:18:52.532785 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-3f751a0e-3430-49e7-97da-58327ee57a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:18:52.626533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:52.646010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 55/109] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:18:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 26 10:18:52.654864 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaddd2a-1cd3-4e0d-8dc0-f545d680b512 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:52.713179 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20ef96-264a-47d4-a777-57e5b13adcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:52.714122 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20ef96-264a-47d4-a777-57e5b13adcec None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:52.714542 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20ef96-264a-47d4-a777-57e5b13adcec None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:52.715021 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20ef96-264a-47d4-a777-57e5b13adcec None admin] PATH_INFO: `/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:52.717055 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b20ef96-264a-47d4-a777-57e5b13adcec None admin] RBAC: Authorizing `identity:get_project(project_id=f81f2060d8dd4c8a85edacc1547c6b3c)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:52.718687 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b20ef96-264a-47d4-a777-57e5b13adcec None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:52.720477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 55/110] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:18:52 2022] GET /identity/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:52.724517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339ef19-1260-46a5-8f25-fd78c222eea8 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:52.724995 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339ef19-1260-46a5-8f25-fd78c222eea8 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:52.725511 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339ef19-1260-46a5-8f25-fd78c222eea8 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:52.727091 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 56/111] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:18:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:18:52.728262 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20ef96-264a-47d4-a777-57e5b13adcec None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:52.732360 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822e262-4846-4eaa-bbdb-25a1c15fce51 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:52.732797 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822e262-4846-4eaa-bbdb-25a1c15fce51 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:52.733228 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822e262-4846-4eaa-bbdb-25a1c15fce51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:52.760677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-9822e262-4846-4eaa-bbdb-25a1c15fce51 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:18:52.763923 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-9822e262-4846-4eaa-bbdb-25a1c15fce51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:18:52.871417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:52.876880 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20ef96-264a-47d4-a777-57e5b13adcec None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:52.882961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 57/112] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:18:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 10:18:52.913591 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bd380-5122-4e21-a758-d55a331fe496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:52.914575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bd380-5122-4e21-a758-d55a331fe496 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:52.914993 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bd380-5122-4e21-a758-d55a331fe496 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:52.915422 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bd380-5122-4e21-a758-d55a331fe496 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:52.916289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556bd380-5122-4e21-a758-d55a331fe496 None admin] RBAC: Adding query filter params (role.id=4ffa6e25be8941078cca7449c84e50d5, scope.project.id=f81f2060d8dd4c8a85edacc1547c6b3c, user.id=9b1ac2894c834148b68b48b22b199c2d) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:18:52.916732 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556bd380-5122-4e21-a758-d55a331fe496 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:52.918385 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556bd380-5122-4e21-a758-d55a331fe496 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:52.924378 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 56/113] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:18:52 2022] GET /identity/v3/role_assignments?user.id=9b1ac2894c834148b68b48b22b199c2d&scope.project.id=f81f2060d8dd4c8a85edacc1547c6b3c&role.id=4ffa6e25be8941078cca7449c84e50d5 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:53.179629 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eae9f7-c54b-43a3-b62d-91cd1e821d7a None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:53.180271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eae9f7-c54b-43a3-b62d-91cd1e821d7a None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:53.180895 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eae9f7-c54b-43a3-b62d-91cd1e821d7a None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:53.182496 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 58/114] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:18:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:18:53.188769 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e65668-263f-44de-bd2b-6b3221cdbfcb None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:53.189061 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e65668-263f-44de-bd2b-6b3221cdbfcb None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:53.189337 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e65668-263f-44de-bd2b-6b3221cdbfcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:53.216017 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-81e65668-263f-44de-bd2b-6b3221cdbfcb None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:18:53.217276 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-81e65668-263f-44de-bd2b-6b3221cdbfcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:18:53.219572 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eae9f7-c54b-43a3-b62d-91cd1e821d7a None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:53.319669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 57/115] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:18:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:18:53.321312 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34392556-915e-42ee-a98b-e0dfb07cdacb None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:53.321803 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34392556-915e-42ee-a98b-e0dfb07cdacb None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:53.322312 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34392556-915e-42ee-a98b-e0dfb07cdacb None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:53.324142 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 58/116] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:18:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:18:53.326821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34392556-915e-42ee-a98b-e0dfb07cdacb None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:53.375352 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eae9f7-c54b-43a3-b62d-91cd1e821d7a None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:53.398362 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:53.414189 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ccb30-ffbf-4ed1-a798-b610ef9d748f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:53.415077 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ccb30-ffbf-4ed1-a798-b610ef9d748f None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:53.416017 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ccb30-ffbf-4ed1-a798-b610ef9d748f None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:53.416452 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ccb30-ffbf-4ed1-a798-b610ef9d748f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:53.439743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ccb30-ffbf-4ed1-a798-b610ef9d748f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:18:53.460757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-2a4ccb30-ffbf-4ed1-a798-b610ef9d748f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 10:18:53.460757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:18:53.460757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:18:53.460757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:18:53.460757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:18:53.460757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:18:53.460757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:18:53.460757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:18:53.460757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:18:53.460757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:18:53.460757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:18:53.460757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:18:53.460757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:18:53.460757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:18:53.460757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:18:53.460757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:18:53.461647 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:18:53.461647 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:18:53.461647 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:18:53.461647 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:18:53.461647 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:18:53.461647 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:18:53.461647 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:18:53.461647 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:18:53.461647 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:18:53.461647 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:18:53.461647 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:18:53.461647 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:18:53.461647 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:18:53.461647 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:18:53.461647 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:18:53.461647 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:18:53.462653 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:18:53.462653 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:18:53.462653 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:18:53.462653 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:18:53.462653 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:18:53.462653 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:18:53.462653 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:18:53.462653 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:18:53.462653 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:18:53.462653 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:18:53.462653 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 10:18:53.462653 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:18:53.462653 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 59/117] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 10:18:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:18:53.463517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5c054-9014-400c-90bc-0e778b8bb378 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:53.463790 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5c054-9014-400c-90bc-0e778b8bb378 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:53.464590 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5c054-9014-400c-90bc-0e778b8bb378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:53.467163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:53.478925 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34392556-915e-42ee-a98b-e0dfb07cdacb None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:53.507038 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-bba5c054-9014-400c-90bc-0e778b8bb378 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:18:53.508258 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-bba5c054-9014-400c-90bc-0e778b8bb378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:18:53.516123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7198a4f-c47b-456e-9f6f-2b9844c3ebbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:53.516945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7198a4f-c47b-456e-9f6f-2b9844c3ebbb None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:53.517363 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7198a4f-c47b-456e-9f6f-2b9844c3ebbb None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:53.517795 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7198a4f-c47b-456e-9f6f-2b9844c3ebbb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:53.528715 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7198a4f-c47b-456e-9f6f-2b9844c3ebbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:18:53.533812 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-a7198a4f-c47b-456e-9f6f-2b9844c3ebbb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:18:53.533812 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:18:53.533812 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:18:53.533812 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:18:53.533812 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:18:53.533812 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:18:53.533812 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:18:53.533812 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:18:53.533812 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:18:53.533812 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:18:53.533812 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:18:53.533812 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:18:53.533812 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:18:53.533812 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:18:53.533812 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:18:53.533812 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:18:53.534847 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:18:53.534847 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:18:53.534847 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:18:53.534847 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:18:53.534847 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:18:53.534847 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:18:53.534847 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:18:53.534847 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:18:53.534847 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:18:53.534847 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:18:53.534847 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:18:53.534847 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:18:53.534847 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:18:53.534847 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:18:53.534847 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:18:53.534847 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:18:53.535725 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:18:53.535725 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:18:53.535725 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:18:53.535725 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:18:53.535725 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:18:53.535725 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:18:53.535725 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:18:53.535725 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:18:53.535725 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:18:53.535725 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:18:53.535725 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:18:53.535725 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:18:53.537459 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 59/118] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:18:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:18:53.542990 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7198a4f-c47b-456e-9f6f-2b9844c3ebbb None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:53.626361 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 60/119] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:18:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:18:53.627099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba5c054-9014-400c-90bc-0e778b8bb378 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:53.710992 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7198a4f-c47b-456e-9f6f-2b9844c3ebbb None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:53.785289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba49702c-7ca2-4099-90ba-a2474dff37ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:53.785906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba49702c-7ca2-4099-90ba-a2474dff37ed None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:53.786247 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba49702c-7ca2-4099-90ba-a2474dff37ed None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:53.786529 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba49702c-7ca2-4099-90ba-a2474dff37ed None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:53.787200 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba49702c-7ca2-4099-90ba-a2474dff37ed None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:18:53.787502 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba49702c-7ca2-4099-90ba-a2474dff37ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:53.789239 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba49702c-7ca2-4099-90ba-a2474dff37ed None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:53.805491 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 60/120] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:18:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:53.806350 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba49702c-7ca2-4099-90ba-a2474dff37ed None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:53.834286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba5c054-9014-400c-90bc-0e778b8bb378 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:53.843017 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:53.850360 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:53.871824 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f8c79-ec52-42b9-8d54-04fc1649625f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:53.874233 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f8c79-ec52-42b9-8d54-04fc1649625f None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:53.874545 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f8c79-ec52-42b9-8d54-04fc1649625f None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:53.874823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f8c79-ec52-42b9-8d54-04fc1649625f None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:53.875504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f8c79-ec52-42b9-8d54-04fc1649625f None admin] RBAC: Adding query filter params (name=member) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:18:53.875812 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f8c79-ec52-42b9-8d54-04fc1649625f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:53.877209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f8c79-ec52-42b9-8d54-04fc1649625f None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:53.890195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 61/121] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 10:18:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:53.891048 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f8c79-ec52-42b9-8d54-04fc1649625f None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:54.009712 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba49702c-7ca2-4099-90ba-a2474dff37ed None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:54.029604 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f8c79-ec52-42b9-8d54-04fc1649625f None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:54.053161 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8380a643-4331-4183-8073-f1f92420a9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:54.054396 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380a643-4331-4183-8073-f1f92420a9b8 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:54.055005 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380a643-4331-4183-8073-f1f92420a9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:54.055478 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380a643-4331-4183-8073-f1f92420a9b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:54.067038 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdce438-c0f9-4a00-82e1-83fbcaa744c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:54.067673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdce438-c0f9-4a00-82e1-83fbcaa744c2 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:54.068004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdce438-c0f9-4a00-82e1-83fbcaa744c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:54.068290 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdce438-c0f9-4a00-82e1-83fbcaa744c2 None admin] PATH_INFO: `/v3/users/4cf56ce5f5694a0499199ccd4ee6ab75` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:54.069427 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8380a643-4331-4183-8073-f1f92420a9b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:18:54.069531 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdce438-c0f9-4a00-82e1-83fbcaa744c2 None admin] RBAC: Authorizing `identity:get_user(user_id=4cf56ce5f5694a0499199ccd4ee6ab75)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:54.070972 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdce438-c0f9-4a00-82e1-83fbcaa744c2 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:54.072563 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 62/122] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:18:53 2022] GET /identity/v3/users/4cf56ce5f5694a0499199ccd4ee6ab75 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:54.073512 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdce438-c0f9-4a00-82e1-83fbcaa744c2 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:54.076510 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-8380a643-4331-4183-8073-f1f92420a9b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:18:54.076510 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:18:54.076510 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:18:54.076510 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:18:54.076510 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:18:54.076510 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:18:54.076510 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:18:54.076510 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:18:54.076510 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:18:54.076510 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:18:54.076510 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:18:54.076510 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:18:54.076510 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:18:54.076510 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:18:54.076510 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:18:54.076510 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:18:54.077802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:18:54.077802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:18:54.077802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:18:54.077802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:18:54.077802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:18:54.077802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:18:54.077802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:18:54.077802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:18:54.077802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:18:54.077802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:18:54.077802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:18:54.077802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:18:54.077802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:18:54.077802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:18:54.077802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:18:54.077802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:18:54.078707 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:18:54.078707 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:18:54.078707 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:18:54.078707 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:18:54.078707 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:18:54.078707 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:18:54.078707 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:18:54.078707 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:18:54.078707 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:18:54.078707 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:18:54.078707 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:18:54.078707 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:18:54.081857 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 61/123] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:18:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:18:54.086956 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8380a643-4331-4183-8073-f1f92420a9b8 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:54.125152 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:54.187851 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:54.206212 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdce438-c0f9-4a00-82e1-83fbcaa744c2 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:54.215483 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8380a643-4331-4183-8073-f1f92420a9b8 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:54.252340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ee922-9e57-40f8-b5b8-5f25236d1cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:54.252869 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6416dfbc-cab0-4f14-91b1-6ccd43fdb200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:54.253506 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416dfbc-cab0-4f14-91b1-6ccd43fdb200 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:54.253764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ee922-9e57-40f8-b5b8-5f25236d1cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:54.253865 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416dfbc-cab0-4f14-91b1-6ccd43fdb200 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:54.254055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416dfbc-cab0-4f14-91b1-6ccd43fdb200 None admin] PATH_INFO: `/v3/users/4cf56ce5f5694a0499199ccd4ee6ab75` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:54.254524 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ee922-9e57-40f8-b5b8-5f25236d1cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:54.254979 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ee922-9e57-40f8-b5b8-5f25236d1cd2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:54.255293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6416dfbc-cab0-4f14-91b1-6ccd43fdb200 None admin] RBAC: Authorizing `identity:get_user(user_id=4cf56ce5f5694a0499199ccd4ee6ab75)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:54.256957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6416dfbc-cab0-4f14-91b1-6ccd43fdb200 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:54.259377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 63/124] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:18:54 2022] GET /identity/v3/users/4cf56ce5f5694a0499199ccd4ee6ab75 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:54.262884 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6416dfbc-cab0-4f14-91b1-6ccd43fdb200 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:54.270762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ee922-9e57-40f8-b5b8-5f25236d1cd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:18:54.276955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-937ee922-9e57-40f8-b5b8-5f25236d1cd2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:18:54.276955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:18:54.276955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:18:54.276955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:18:54.276955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:18:54.276955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:18:54.276955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:18:54.276955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:18:54.276955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:18:54.276955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:18:54.276955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:18:54.276955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:18:54.276955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:18:54.276955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:18:54.276955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:18:54.276955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:18:54.277852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:18:54.277852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:18:54.277852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:18:54.277852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:18:54.277852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:18:54.277852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:18:54.277852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:18:54.277852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:18:54.277852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:18:54.277852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:18:54.277852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:18:54.277852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:18:54.277852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:18:54.277852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:18:54.277852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:18:54.277852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:18:54.278680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:18:54.278680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:18:54.278680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:18:54.278680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:18:54.278680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:18:54.278680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:18:54.278680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:18:54.278680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:18:54.278680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:18:54.278680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:18:54.278680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:18:54.278680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:18:54.282093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 62/125] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:18:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:18:54.283173 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ee922-9e57-40f8-b5b8-5f25236d1cd2 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:54.382911 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6416dfbc-cab0-4f14-91b1-6ccd43fdb200 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:54.394463 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:54.412293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ee922-9e57-40f8-b5b8-5f25236d1cd2 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:54.423979 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c9765-6bdb-4df8-8bc3-8df0577468b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:54.424609 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c9765-6bdb-4df8-8bc3-8df0577468b2 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:54.424887 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c9765-6bdb-4df8-8bc3-8df0577468b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:54.425161 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c9765-6bdb-4df8-8bc3-8df0577468b2 None admin] PATH_INFO: `/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:54.426442 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400c9765-6bdb-4df8-8bc3-8df0577468b2 None admin] RBAC: Authorizing `identity:get_project(project_id=f81f2060d8dd4c8a85edacc1547c6b3c)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:54.428069 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400c9765-6bdb-4df8-8bc3-8df0577468b2 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:54.435646 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 64/126] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:18:54 2022] GET /identity/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:54.435646 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c9765-6bdb-4df8-8bc3-8df0577468b2 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:54.467989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d16cb3-601e-4432-a6c8-f0ca9d2bc054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:54.469556 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d16cb3-601e-4432-a6c8-f0ca9d2bc054 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:54.470070 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d16cb3-601e-4432-a6c8-f0ca9d2bc054 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:54.470571 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d16cb3-601e-4432-a6c8-f0ca9d2bc054 None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:54.471697 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d16cb3-601e-4432-a6c8-f0ca9d2bc054 None admin] RBAC: Adding query filter params (name=member) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:18:54.473091 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d16cb3-601e-4432-a6c8-f0ca9d2bc054 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:54.474762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d16cb3-601e-4432-a6c8-f0ca9d2bc054 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:54.476397 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:54.488880 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 63/127] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 10:18:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:54.490710 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d16cb3-601e-4432-a6c8-f0ca9d2bc054 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:54.558309 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c9765-6bdb-4df8-8bc3-8df0577468b2 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:54.600068 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fbfafb-c08a-4c33-aadf-89a0dd916f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:54.601108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbfafb-c08a-4c33-aadf-89a0dd916f2f None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:54.601628 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbfafb-c08a-4c33-aadf-89a0dd916f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:54.602550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbfafb-c08a-4c33-aadf-89a0dd916f2f None admin] PATH_INFO: `/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:54.604161 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fbfafb-c08a-4c33-aadf-89a0dd916f2f None admin] RBAC: Authorizing `identity:get_project(project_id=f81f2060d8dd4c8a85edacc1547c6b3c)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:54.606651 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fbfafb-c08a-4c33-aadf-89a0dd916f2f None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:54.609513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 65/128] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:18:54 2022] GET /identity/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:54.610543 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fbfafb-c08a-4c33-aadf-89a0dd916f2f None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:54.619511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d16cb3-601e-4432-a6c8-f0ca9d2bc054 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:54.660488 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5216f62f-d34f-47e4-a827-4f64c36a9fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:54.661106 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216f62f-d34f-47e4-a827-4f64c36a9fca None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:54.661393 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216f62f-d34f-47e4-a827-4f64c36a9fca None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:54.661668 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216f62f-d34f-47e4-a827-4f64c36a9fca None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:54.662373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5216f62f-d34f-47e4-a827-4f64c36a9fca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:18:54.662734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5216f62f-d34f-47e4-a827-4f64c36a9fca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:54.664668 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5216f62f-d34f-47e4-a827-4f64c36a9fca None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:54.669561 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:54.676207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 64/129] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:18:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:54.677010 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5216f62f-d34f-47e4-a827-4f64c36a9fca None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:54.734715 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fbfafb-c08a-4c33-aadf-89a0dd916f2f None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:54.735088 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:54.770559 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f3044-cb51-4f97-b8cb-ed9ffa1ee466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:54.771630 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f3044-cb51-4f97-b8cb-ed9ffa1ee466 None admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:54.771927 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f3044-cb51-4f97-b8cb-ed9ffa1ee466 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:54.772235 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f3044-cb51-4f97-b8cb-ed9ffa1ee466 None admin] PATH_INFO: `/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c/users/4cf56ce5f5694a0499199ccd4ee6ab75/roles/d1109db57b004dd9a4f84a634ff76205` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:54.774583 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f3044-cb51-4f97-b8cb-ed9ffa1ee466 None admin] RBAC: Authorizing `identity:create_grant(project_id=f81f2060d8dd4c8a85edacc1547c6b3c, user_id=4cf56ce5f5694a0499199ccd4ee6ab75, role_id=d1109db57b004dd9a4f84a634ff76205)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:54.776045 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f3044-cb51-4f97-b8cb-ed9ffa1ee466 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:54.784413 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5216f62f-d34f-47e4-a827-4f64c36a9fca None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:54.790285 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 66/130] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 10:18:54 2022] PUT /identity/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c/users/4cf56ce5f5694a0499199ccd4ee6ab75/roles/d1109db57b004dd9a4f84a634ff76205 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:18:54.799413 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f3044-cb51-4f97-b8cb-ed9ffa1ee466 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:54.831423 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d93b5-dcba-4cb5-a6d9-605085cb6040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:54.832116 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d93b5-dcba-4cb5-a6d9-605085cb6040 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:54.833109 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d93b5-dcba-4cb5-a6d9-605085cb6040 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:54.833436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d93b5-dcba-4cb5-a6d9-605085cb6040 None admin] PATH_INFO: `/v3/users/4cf56ce5f5694a0499199ccd4ee6ab75` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:54.834965 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d93b5-dcba-4cb5-a6d9-605085cb6040 None admin] RBAC: Authorizing `identity:get_user(user_id=4cf56ce5f5694a0499199ccd4ee6ab75)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:54.837380 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d93b5-dcba-4cb5-a6d9-605085cb6040 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:54.840449 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 65/131] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:18:54 2022] GET /identity/v3/users/4cf56ce5f5694a0499199ccd4ee6ab75 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:54.842175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77b85b-7d92-4216-9a9b-bc46200b390b None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:54.842460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77b85b-7d92-4216-9a9b-bc46200b390b None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:54.842980 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77b85b-7d92-4216-9a9b-bc46200b390b None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:54.845432 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 66/132] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:18:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:18:54.846264 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca77b85b-7d92-4216-9a9b-bc46200b390b None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:54.952834 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f3044-cb51-4f97-b8cb-ed9ffa1ee466 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:54.957066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:54.996579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca77b85b-7d92-4216-9a9b-bc46200b390b None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:55.004714 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e24a21-9765-4b68-bcea-d12bc1b0ea08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:55.005399 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e24a21-9765-4b68-bcea-d12bc1b0ea08 None admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:55.005670 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e24a21-9765-4b68-bcea-d12bc1b0ea08 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:55.005946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e24a21-9765-4b68-bcea-d12bc1b0ea08 None admin] PATH_INFO: `/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c/users/4cf56ce5f5694a0499199ccd4ee6ab75/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:55.008031 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e24a21-9765-4b68-bcea-d12bc1b0ea08 None admin] RBAC: Authorizing `identity:create_grant(project_id=f81f2060d8dd4c8a85edacc1547c6b3c, user_id=4cf56ce5f5694a0499199ccd4ee6ab75, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:55.009621 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e24a21-9765-4b68-bcea-d12bc1b0ea08 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:55.029750 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:55.038070 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 67/133] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 10:18:54 2022] PUT /identity/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c/users/4cf56ce5f5694a0499199ccd4ee6ab75/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:18:55.038989 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e24a21-9765-4b68-bcea-d12bc1b0ea08 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:55.040969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2424a15-bb7b-45d2-aa02-82f98df90443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:55.041893 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2424a15-bb7b-45d2-aa02-82f98df90443 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:55.042505 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2424a15-bb7b-45d2-aa02-82f98df90443 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:55.043009 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2424a15-bb7b-45d2-aa02-82f98df90443 None admin] PATH_INFO: `/v3/users/7ecff1fbd79248b7bcd1e69035be3515` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:55.044683 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2424a15-bb7b-45d2-aa02-82f98df90443 None admin] RBAC: Authorizing `identity:get_user(user_id=7ecff1fbd79248b7bcd1e69035be3515)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:55.046498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2424a15-bb7b-45d2-aa02-82f98df90443 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:55.048540 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 67/134] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:18:54 2022] GET /identity/v3/users/7ecff1fbd79248b7bcd1e69035be3515 => generated 247 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:55.049954 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db7d03-b1dc-4d67-970e-09a08538fc79 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:55.050569 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db7d03-b1dc-4d67-970e-09a08538fc79 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:55.051048 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db7d03-b1dc-4d67-970e-09a08538fc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:55.072283 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-51db7d03-b1dc-4d67-970e-09a08538fc79 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:18:55.074067 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-51db7d03-b1dc-4d67-970e-09a08538fc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:18:55.153395 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 68/135] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:18:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:18:55.154856 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db7d03-b1dc-4d67-970e-09a08538fc79 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:55.179390 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e24a21-9765-4b68-bcea-d12bc1b0ea08 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:55.227887 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0fefa-1e01-4b53-abfd-b26e4b9c4267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:55.228894 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0fefa-1e01-4b53-abfd-b26e4b9c4267 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:55.229378 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0fefa-1e01-4b53-abfd-b26e4b9c4267 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:55.229906 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0fefa-1e01-4b53-abfd-b26e4b9c4267 None admin] PATH_INFO: `/v3/projects/51fc4712d2454146b6103cf60db3fb7a` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:55.231304 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0fefa-1e01-4b53-abfd-b26e4b9c4267 None admin] RBAC: Authorizing `identity:get_project(project_id=51fc4712d2454146b6103cf60db3fb7a)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:55.233029 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0fefa-1e01-4b53-abfd-b26e4b9c4267 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:55.234920 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 68/136] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:18:55 2022] GET /identity/v3/projects/51fc4712d2454146b6103cf60db3fb7a => generated 312 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:55.240844 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0fefa-1e01-4b53-abfd-b26e4b9c4267 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:55.282082 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:55.283759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db7d03-b1dc-4d67-970e-09a08538fc79 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:55.318535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:55.337900 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f39f5-613c-4e3a-983a-6146af501d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:55.339038 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f39f5-613c-4e3a-983a-6146af501d05 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:55.339322 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f39f5-613c-4e3a-983a-6146af501d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:55.339864 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f39f5-613c-4e3a-983a-6146af501d05 None admin] PATH_INFO: `/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:55.341403 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408f39f5-613c-4e3a-983a-6146af501d05 None admin] RBAC: Authorizing `identity:get_project(project_id=f81f2060d8dd4c8a85edacc1547c6b3c)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:55.343585 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408f39f5-613c-4e3a-983a-6146af501d05 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:55.346293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 69/137] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:18:55 2022] GET /identity/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:55.354934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f39f5-613c-4e3a-983a-6146af501d05 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:55.393008 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0fefa-1e01-4b53-abfd-b26e4b9c4267 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:55.468571 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c4b39-d1da-4361-8178-256463c48b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:55.469409 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c4b39-d1da-4361-8178-256463c48b17 None admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:55.469702 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c4b39-d1da-4361-8178-256463c48b17 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:55.469974 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c4b39-d1da-4361-8178-256463c48b17 None admin] PATH_INFO: `/v3/projects/51fc4712d2454146b6103cf60db3fb7a/users/4cf56ce5f5694a0499199ccd4ee6ab75/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:55.472678 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97c4b39-d1da-4361-8178-256463c48b17 None admin] RBAC: Authorizing `identity:create_grant(project_id=51fc4712d2454146b6103cf60db3fb7a, user_id=4cf56ce5f5694a0499199ccd4ee6ab75, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:55.474171 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97c4b39-d1da-4361-8178-256463c48b17 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:55.503461 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 69/138] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 10:18:55 2022] PUT /identity/v3/projects/51fc4712d2454146b6103cf60db3fb7a/users/4cf56ce5f5694a0499199ccd4ee6ab75/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:18:55.553451 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f39f5-613c-4e3a-983a-6146af501d05 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:55.604840 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818e5e5-6810-4094-8128-59c100e5748c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:55.605539 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818e5e5-6810-4094-8128-59c100e5748c None admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:55.605820 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818e5e5-6810-4094-8128-59c100e5748c None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:55.606121 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818e5e5-6810-4094-8128-59c100e5748c None admin] PATH_INFO: `/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c/users/7ecff1fbd79248b7bcd1e69035be3515/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:55.611533 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f818e5e5-6810-4094-8128-59c100e5748c None admin] RBAC: Authorizing `identity:create_grant(project_id=f81f2060d8dd4c8a85edacc1547c6b3c, user_id=7ecff1fbd79248b7bcd1e69035be3515, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:55.613040 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f818e5e5-6810-4094-8128-59c100e5748c None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:55.634146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 70/139] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 10:18:55 2022] PUT /identity/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c/users/7ecff1fbd79248b7bcd1e69035be3515/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:18:55.687702 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c4b39-d1da-4361-8178-256463c48b17 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:55.811223 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:55.834326 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c4b39-d1da-4361-8178-256463c48b17 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:55.871699 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de349670-01f6-4913-b269-2a9feb3a478e None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:55.872205 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de349670-01f6-4913-b269-2a9feb3a478e None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:55.872277 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de349670-01f6-4913-b269-2a9feb3a478e None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:55.874246 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 71/140] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:18:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:18:55.881731 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404ef0a-d6b2-4a1e-bd55-5f80d0a42355 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:55.882083 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404ef0a-d6b2-4a1e-bd55-5f80d0a42355 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:55.882311 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404ef0a-d6b2-4a1e-bd55-5f80d0a42355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:55.893203 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360518a-a555-430f-93b7-a8e199766b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:55.893867 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360518a-a555-430f-93b7-a8e199766b14 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:55.894187 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360518a-a555-430f-93b7-a8e199766b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:55.894465 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360518a-a555-430f-93b7-a8e199766b14 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:55.896512 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8360518a-a555-430f-93b7-a8e199766b14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:55.897986 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8360518a-a555-430f-93b7-a8e199766b14 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:55.905597 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 70/141] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:18:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:55.911178 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360518a-a555-430f-93b7-a8e199766b14 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:55.923294 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-1404ef0a-d6b2-4a1e-bd55-5f80d0a42355 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:18:55.924509 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-1404ef0a-d6b2-4a1e-bd55-5f80d0a42355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:18:56.010387 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:56.047829 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 72/142] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:18:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 10:18:56.065559 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360518a-a555-430f-93b7-a8e199766b14 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:56.107988 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f261fe-798f-49bc-ba7e-9e0bd0882f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:56.108608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f261fe-798f-49bc-ba7e-9e0bd0882f99 None admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:56.108891 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f261fe-798f-49bc-ba7e-9e0bd0882f99 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:56.109166 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f261fe-798f-49bc-ba7e-9e0bd0882f99 None admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:56.109828 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f261fe-798f-49bc-ba7e-9e0bd0882f99 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:56.111306 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f261fe-798f-49bc-ba7e-9e0bd0882f99 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:56.146747 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 71/143] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 10:18:55 2022] POST /identity/v3/users => generated 283 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:18:56.571557 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404ef0a-d6b2-4a1e-bd55-5f80d0a42355 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:56.685694 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404ef0a-d6b2-4a1e-bd55-5f80d0a42355 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:56.722642 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222b2b0-1b37-4a1e-a6f1-b2b8af6f9ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:56.723300 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222b2b0-1b37-4a1e-a6f1-b2b8af6f9ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:56.723579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222b2b0-1b37-4a1e-a6f1-b2b8af6f9ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:56.723849 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222b2b0-1b37-4a1e-a6f1-b2b8af6f9ea6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:56.734607 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a222b2b0-1b37-4a1e-a6f1-b2b8af6f9ea6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:18:56.739512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-a222b2b0-1b37-4a1e-a6f1-b2b8af6f9ea6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 10:18:56.739512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:18:56.739512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:18:56.739512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:18:56.739512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:18:56.739512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:18:56.739512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:18:56.739512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:18:56.739512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:18:56.739512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:18:56.739512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:18:56.739512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:18:56.739512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:18:56.739512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:18:56.739512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:18:56.739512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:18:56.740386 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:18:56.740386 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:18:56.740386 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:18:56.740386 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:18:56.740386 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:18:56.740386 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:18:56.740386 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:18:56.740386 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:18:56.740386 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:18:56.740386 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:18:56.740386 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:18:56.740386 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:18:56.740386 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:18:56.740386 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:18:56.740386 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:18:56.740386 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:18:56.741147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:18:56.741147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:18:56.741147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:18:56.741147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:18:56.741147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:18:56.741147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:18:56.741147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:18:56.741147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:18:56.741147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:18:56.741147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:18:56.741147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 10:18:56.741147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:18:56.741147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 73/144] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:18:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:18:56.748440 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f261fe-798f-49bc-ba7e-9e0bd0882f99 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:56.784631 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:56.924693 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f261fe-798f-49bc-ba7e-9e0bd0882f99 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:56.969107 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c09afd-3659-4287-8201-78e0cc2339fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:56.969731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c09afd-3659-4287-8201-78e0cc2339fc None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:56.970012 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c09afd-3659-4287-8201-78e0cc2339fc None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:56.970313 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c09afd-3659-4287-8201-78e0cc2339fc None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:56.970996 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c09afd-3659-4287-8201-78e0cc2339fc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:18:56.971303 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c09afd-3659-4287-8201-78e0cc2339fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:56.972689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c09afd-3659-4287-8201-78e0cc2339fc None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:56.985349 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 72/145] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 10:18:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:56.991280 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222b2b0-1b37-4a1e-a6f1-b2b8af6f9ea6 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:57.083566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:57.177831 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222b2b0-1b37-4a1e-a6f1-b2b8af6f9ea6 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:57.235046 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c631f-ee99-453f-8ade-f1e855dbcd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:57.236017 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c631f-ee99-453f-8ade-f1e855dbcd2e None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:57.236487 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c631f-ee99-453f-8ade-f1e855dbcd2e None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:57.236948 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c631f-ee99-453f-8ade-f1e855dbcd2e None admin] PATH_INFO: `/v3/users/9b1ac2894c834148b68b48b22b199c2d` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:57.242694 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999c631f-ee99-453f-8ade-f1e855dbcd2e None admin] RBAC: Authorizing `identity:get_user(user_id=9b1ac2894c834148b68b48b22b199c2d)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:57.244366 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999c631f-ee99-453f-8ade-f1e855dbcd2e None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:57.246202 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 74/146] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:18:56 2022] GET /identity/v3/users/9b1ac2894c834148b68b48b22b199c2d => generated 289 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:57.252439 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c09afd-3659-4287-8201-78e0cc2339fc None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:57.360062 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:57.393065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c09afd-3659-4287-8201-78e0cc2339fc None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:57.435258 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db88ce-a9db-450a-9ca0-440e8cf7b38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:57.435825 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db88ce-a9db-450a-9ca0-440e8cf7b38e None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:57.436106 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db88ce-a9db-450a-9ca0-440e8cf7b38e None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:57.436384 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db88ce-a9db-450a-9ca0-440e8cf7b38e None admin] PATH_INFO: `/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:57.439258 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db88ce-a9db-450a-9ca0-440e8cf7b38e None admin] RBAC: Authorizing `identity:get_project(project_id=f81f2060d8dd4c8a85edacc1547c6b3c)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:57.440744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db88ce-a9db-450a-9ca0-440e8cf7b38e None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:57.442748 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 73/147] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:18:57 2022] GET /identity/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:57.448425 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c631f-ee99-453f-8ade-f1e855dbcd2e None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:57.560687 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7b0b1-83e6-4c9c-91e6-5b0d2f2327cb None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:57.561025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7b0b1-83e6-4c9c-91e6-5b0d2f2327cb None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:57.561252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7b0b1-83e6-4c9c-91e6-5b0d2f2327cb None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:57.562887 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 74/148] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:18:57.579022 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db86a3d-7110-4149-935e-efd582db8332 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:57.579565 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db86a3d-7110-4149-935e-efd582db8332 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:57.579901 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db86a3d-7110-4149-935e-efd582db8332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:57.596776 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c631f-ee99-453f-8ade-f1e855dbcd2e None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:57.613685 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:57.621165 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-1db86a3d-7110-4149-935e-efd582db8332 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:18:57.622473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-1db86a3d-7110-4149-935e-efd582db8332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:18:57.666753 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda43f3b-d59e-48be-88e7-9ff0fa5abbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:57.667786 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda43f3b-d59e-48be-88e7-9ff0fa5abbd9 None admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:57.668253 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda43f3b-d59e-48be-88e7-9ff0fa5abbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:57.668708 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda43f3b-d59e-48be-88e7-9ff0fa5abbd9 None admin] PATH_INFO: `/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c/users/9b1ac2894c834148b68b48b22b199c2d/roles/4ffa6e25be8941078cca7449c84e50d5` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:57.671597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda43f3b-d59e-48be-88e7-9ff0fa5abbd9 None admin] RBAC: Authorizing `identity:create_grant(project_id=f81f2060d8dd4c8a85edacc1547c6b3c, user_id=9b1ac2894c834148b68b48b22b199c2d, role_id=4ffa6e25be8941078cca7449c84e50d5)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:57.673271 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda43f3b-d59e-48be-88e7-9ff0fa5abbd9 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:57.693723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 75/149] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 10:18:57 2022] PUT /identity/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c/users/9b1ac2894c834148b68b48b22b199c2d/roles/4ffa6e25be8941078cca7449c84e50d5 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:18:57.757782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 75/150] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:18:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 10:18:58.318110 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba607a-1935-45a5-8b62-2dd9fb32c7d3 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:58.319121 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba607a-1935-45a5-8b62-2dd9fb32c7d3 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:58.319617 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba607a-1935-45a5-8b62-2dd9fb32c7d3 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:58.321159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 76/151] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:18:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:18:58.327116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e87f12-1b7e-420e-8812-aff0fd554e73 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:58.327656 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e87f12-1b7e-420e-8812-aff0fd554e73 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:58.328137 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e87f12-1b7e-420e-8812-aff0fd554e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:58.355022 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-37e87f12-1b7e-420e-8812-aff0fd554e73 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:18:58.356515 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-37e87f12-1b7e-420e-8812-aff0fd554e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:18:58.377521 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecba607a-1935-45a5-8b62-2dd9fb32c7d3 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:58.437491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 76/152] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:18:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:18:58.541806 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecba607a-1935-45a5-8b62-2dd9fb32c7d3 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:58.589552 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e33e9c-1651-4488-be60-9c6f5629cedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:58.590142 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e33e9c-1651-4488-be60-9c6f5629cedc None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:58.590423 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e33e9c-1651-4488-be60-9c6f5629cedc None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:58.590696 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e33e9c-1651-4488-be60-9c6f5629cedc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:58.608032 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e33e9c-1651-4488-be60-9c6f5629cedc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:18:58.613017 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-95e33e9c-1651-4488-be60-9c6f5629cedc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 10:18:58.613017 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:18:58.613017 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:18:58.613017 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:18:58.613017 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:18:58.613017 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:18:58.613017 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:18:58.613017 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:18:58.613017 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:18:58.613017 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:18:58.613017 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:18:58.613017 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:18:58.613017 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:18:58.613017 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:18:58.613017 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:18:58.613017 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:18:58.613942 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:18:58.613942 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:18:58.613942 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:18:58.613942 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:18:58.613942 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:18:58.613942 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:18:58.613942 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:18:58.613942 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:18:58.613942 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:18:58.613942 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:18:58.613942 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:18:58.613942 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:18:58.613942 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:18:58.613942 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:18:58.613942 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:18:58.613942 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:18:58.615133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:18:58.615133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:18:58.615133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:18:58.615133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:18:58.615133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:18:58.615133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:18:58.615133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:18:58.615133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:18:58.615133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:18:58.615133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:18:58.615133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 10:18:58.615133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:18:58.615133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 77/153] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 10:18:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:18:58.620240 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e87f12-1b7e-420e-8812-aff0fd554e73 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:58.634927 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:58.785926 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e87f12-1b7e-420e-8812-aff0fd554e73 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:58.836643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc9efe-a392-4ee8-9824-363c5dd062e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:58.837263 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc9efe-a392-4ee8-9824-363c5dd062e4 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:58.837542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc9efe-a392-4ee8-9824-363c5dd062e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:58.837817 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc9efe-a392-4ee8-9824-363c5dd062e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:58.838517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc9efe-a392-4ee8-9824-363c5dd062e4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:18:58.838825 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc9efe-a392-4ee8-9824-363c5dd062e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:58.840267 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc9efe-a392-4ee8-9824-363c5dd062e4 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:58.849201 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 77/154] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:18:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:58.854984 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e33e9c-1651-4488-be60-9c6f5629cedc None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:58.891939 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:58.977329 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4474e4b-e4ef-42ac-9399-3527719bf4ec None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:58.977677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4474e4b-e4ef-42ac-9399-3527719bf4ec None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:58.977893 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4474e4b-e4ef-42ac-9399-3527719bf4ec None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:58.979278 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 78/155] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:18:58.987835 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e33e9c-1651-4488-be60-9c6f5629cedc None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:58.989695 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ff36f-3a54-462e-8c29-7b7d30b7bc11 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:58.989978 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ff36f-3a54-462e-8c29-7b7d30b7bc11 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:58.990279 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ff36f-3a54-462e-8c29-7b7d30b7bc11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:59.026175 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-d16ff36f-3a54-462e-8c29-7b7d30b7bc11 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:18:59.027661 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-d16ff36f-3a54-462e-8c29-7b7d30b7bc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:18:59.093544 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f6240a-eeab-4ba6-ae8f-766129c46abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:59.094138 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6240a-eeab-4ba6-ae8f-766129c46abf None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:59.094418 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6240a-eeab-4ba6-ae8f-766129c46abf None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:59.094692 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6240a-eeab-4ba6-ae8f-766129c46abf None admin] PATH_INFO: `/v3/users/4cf56ce5f5694a0499199ccd4ee6ab75` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:59.097018 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f6240a-eeab-4ba6-ae8f-766129c46abf None admin] RBAC: Authorizing `identity:get_user(user_id=4cf56ce5f5694a0499199ccd4ee6ab75)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:59.099328 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f6240a-eeab-4ba6-ae8f-766129c46abf None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:59.100955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 78/156] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:18:58 2022] GET /identity/v3/users/4cf56ce5f5694a0499199ccd4ee6ab75 => generated 275 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:59.114573 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f6240a-eeab-4ba6-ae8f-766129c46abf None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:59.184947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:59.193873 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 79/157] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:18:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:18:59.195429 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b15e9-22be-4897-a5f6-d1dd3307b5e0 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:59.195884 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b15e9-22be-4897-a5f6-d1dd3307b5e0 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:59.196303 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b15e9-22be-4897-a5f6-d1dd3307b5e0 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:59.197753 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 80/158] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:18:59.198681 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b15e9-22be-4897-a5f6-d1dd3307b5e0 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:59.314450 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:59.321696 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f6240a-eeab-4ba6-ae8f-766129c46abf None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:59.380818 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b15e9-22be-4897-a5f6-d1dd3307b5e0 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:59.386559 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18938691-8b2d-46a7-8272-ba4655d7b90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:59.387600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18938691-8b2d-46a7-8272-ba4655d7b90f None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:59.390868 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18938691-8b2d-46a7-8272-ba4655d7b90f None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:59.391303 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18938691-8b2d-46a7-8272-ba4655d7b90f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:59.458765 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cdea42-b7d4-44e7-9112-f48bf461ac9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:59.459421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cdea42-b7d4-44e7-9112-f48bf461ac9a None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:59.459704 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cdea42-b7d4-44e7-9112-f48bf461ac9a None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:59.459983 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cdea42-b7d4-44e7-9112-f48bf461ac9a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:59.475457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cdea42-b7d4-44e7-9112-f48bf461ac9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:18:59.481588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-24cdea42-b7d4-44e7-9112-f48bf461ac9a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:18:59.481588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:18:59.481588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:18:59.481588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:18:59.481588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:18:59.481588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:18:59.481588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:18:59.481588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:18:59.481588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:18:59.481588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:18:59.481588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:18:59.481588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:18:59.481588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:18:59.481588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:18:59.481588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:18:59.481588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:18:59.482802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:18:59.482802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:18:59.482802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:18:59.482802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:18:59.482802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:18:59.482802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:18:59.482802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:18:59.482802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:18:59.482802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:18:59.482802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:18:59.482802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:18:59.482802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:18:59.482802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:18:59.482802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:18:59.482802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:18:59.482802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:18:59.483689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:18:59.483689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:18:59.483689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:18:59.483689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:18:59.483689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:18:59.483689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:18:59.483689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:18:59.483689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:18:59.483689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:18:59.483689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:18:59.483689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:18:59.483689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:18:59.483689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 81/159] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:18:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:18:59.485149 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f38f1-d775-497e-b842-6473e6f155c0 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:59.485440 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f38f1-d775-497e-b842-6473e6f155c0 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:59.485783 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f38f1-d775-497e-b842-6473e6f155c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:59.519182 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-b53f38f1-d775-497e-b842-6473e6f155c0 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:18:59.520422 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-b53f38f1-d775-497e-b842-6473e6f155c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:18:59.528284 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18938691-8b2d-46a7-8272-ba4655d7b90f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:59.530782 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18938691-8b2d-46a7-8272-ba4655d7b90f None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:59.621172 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:59.664166 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 82/160] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:18:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:18:59.665025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f38f1-d775-497e-b842-6473e6f155c0 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:59.722355 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:18:59.727443 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 79/161] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:18:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 10:18:59.733154 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18938691-8b2d-46a7-8272-ba4655d7b90f None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:18:59.894053 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f38f1-d775-497e-b842-6473e6f155c0 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:59.921248 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18938691-8b2d-46a7-8272-ba4655d7b90f None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:18:59.959268 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b4a2f5-16b9-4552-89be-fbc529380514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:59.959909 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4a2f5-16b9-4552-89be-fbc529380514 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:59.960183 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4a2f5-16b9-4552-89be-fbc529380514 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:59.960448 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4a2f5-16b9-4552-89be-fbc529380514 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:59.980095 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b4a2f5-16b9-4552-89be-fbc529380514 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:18:59.982699 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b233ed4-2ac7-4dcb-b564-5ce63c09d0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:18:59.983324 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b233ed4-2ac7-4dcb-b564-5ce63c09d0ac None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:18:59.983600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b233ed4-2ac7-4dcb-b564-5ce63c09d0ac None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:18:59.983869 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b233ed4-2ac7-4dcb-b564-5ce63c09d0ac None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:18:59.984543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b233ed4-2ac7-4dcb-b564-5ce63c09d0ac None admin] RBAC: Adding query filter params (name=member) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:18:59.984841 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b233ed4-2ac7-4dcb-b564-5ce63c09d0ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:18:59.984954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-68b4a2f5-16b9-4552-89be-fbc529380514 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:18:59.984954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:18:59.984954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:18:59.984954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:18:59.984954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:18:59.984954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:18:59.984954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:18:59.984954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:18:59.984954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:18:59.984954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:18:59.984954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:18:59.984954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:18:59.984954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:18:59.984954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:18:59.984954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:18:59.984954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:18:59.985744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:18:59.985744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:18:59.985744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:18:59.985744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:18:59.985744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:18:59.985744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:18:59.985744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:18:59.985744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:18:59.985744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:18:59.985744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:18:59.985744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:18:59.985744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:18:59.985744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:18:59.985744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:18:59.985744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:18:59.985744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:18:59.986605 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:18:59.986605 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:18:59.986605 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:18:59.986605 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:18:59.986605 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:18:59.986605 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:18:59.986605 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:18:59.986605 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:18:59.986605 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:18:59.986605 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:18:59.986605 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:18:59.986605 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:18:59.987199 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b233ed4-2ac7-4dcb-b564-5ce63c09d0ac None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:18:59.994486 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 83/162] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:18:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:18:59.994942 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 80/163] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 10:18:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:18:59.995389 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b4a2f5-16b9-4552-89be-fbc529380514 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:00.027385 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b233ed4-2ac7-4dcb-b564-5ce63c09d0ac None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:00.077003 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:00.123911 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:00.162547 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b4a2f5-16b9-4552-89be-fbc529380514 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:00.188385 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b233ed4-2ac7-4dcb-b564-5ce63c09d0ac None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:00.229867 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc7653f-93ed-43fe-acd8-d3906f8b51b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:00.230521 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7653f-93ed-43fe-acd8-d3906f8b51b0 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:00.230798 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7653f-93ed-43fe-acd8-d3906f8b51b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:00.231068 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7653f-93ed-43fe-acd8-d3906f8b51b0 None admin] PATH_INFO: `/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:00.233726 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc7653f-93ed-43fe-acd8-d3906f8b51b0 None admin] RBAC: Authorizing `identity:get_project(project_id=f81f2060d8dd4c8a85edacc1547c6b3c)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:00.235198 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc7653f-93ed-43fe-acd8-d3906f8b51b0 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:00.236828 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 84/164] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:18:59 2022] GET /identity/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:00.239133 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc7653f-93ed-43fe-acd8-d3906f8b51b0 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:00.247061 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e5604-5ab6-461c-bc43-c2b4afa2eb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:00.248342 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e5604-5ab6-461c-bc43-c2b4afa2eb5b None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:00.248614 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e5604-5ab6-461c-bc43-c2b4afa2eb5b None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:00.248879 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e5604-5ab6-461c-bc43-c2b4afa2eb5b None admin] PATH_INFO: `/v3/users/4cf56ce5f5694a0499199ccd4ee6ab75` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:00.250688 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058e5604-5ab6-461c-bc43-c2b4afa2eb5b None admin] RBAC: Authorizing `identity:get_user(user_id=4cf56ce5f5694a0499199ccd4ee6ab75)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:00.252133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058e5604-5ab6-461c-bc43-c2b4afa2eb5b None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:00.253712 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 81/165] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:19:00 2022] GET /identity/v3/users/4cf56ce5f5694a0499199ccd4ee6ab75 => generated 275 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:00.263179 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e5604-5ab6-461c-bc43-c2b4afa2eb5b None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:00.423533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc7653f-93ed-43fe-acd8-d3906f8b51b0 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:00.453110 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e5604-5ab6-461c-bc43-c2b4afa2eb5b None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:00.464929 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:00.505172 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439f7b0b-01c4-4f78-88e0-9c778b2f312a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:00.505790 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f7b0b-01c4-4f78-88e0-9c778b2f312a None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:00.506086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f7b0b-01c4-4f78-88e0-9c778b2f312a None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:00.506361 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f7b0b-01c4-4f78-88e0-9c778b2f312a None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:00.507029 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439f7b0b-01c4-4f78-88e0-9c778b2f312a None admin] RBAC: Adding query filter params (name=member) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:00.507331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439f7b0b-01c4-4f78-88e0-9c778b2f312a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:00.508704 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439f7b0b-01c4-4f78-88e0-9c778b2f312a None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:00.514005 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:00.531266 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 85/166] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 10:19:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:00.532100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439f7b0b-01c4-4f78-88e0-9c778b2f312a None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:00.533866 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116caad-5c86-410a-b161-35929cdb5737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:00.542579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116caad-5c86-410a-b161-35929cdb5737 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:00.542868 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116caad-5c86-410a-b161-35929cdb5737 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:00.543147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116caad-5c86-410a-b161-35929cdb5737 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:00.572114 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116caad-5c86-410a-b161-35929cdb5737 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:00.577589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-d116caad-5c86-410a-b161-35929cdb5737 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:19:00.577589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:00.577589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:00.577589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:00.577589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:00.577589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:00.577589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:00.577589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:00.577589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:00.577589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:00.577589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:00.577589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:00.577589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:00.577589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:00.577589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:00.577589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:00.578520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:00.578520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:00.578520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:00.578520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:00.578520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:00.578520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:00.578520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:00.578520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:00.578520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:00.578520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:00.578520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:00.578520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:00.578520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:00.578520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:00.578520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:00.578520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:00.579677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:00.579677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:00.579677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:00.579677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:00.579677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:00.579677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:00.579677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:00.579677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:00.579677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:00.579677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:00.579677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:19:00.579677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:00.583666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 82/167] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:19:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:00.584460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116caad-5c86-410a-b161-35929cdb5737 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:00.759966 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439f7b0b-01c4-4f78-88e0-9c778b2f312a None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:00.817744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116caad-5c86-410a-b161-35929cdb5737 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:00.827608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2930e-42db-4f08-9bc0-194d6b2b5076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:00.828225 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2930e-42db-4f08-9bc0-194d6b2b5076 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:00.828502 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2930e-42db-4f08-9bc0-194d6b2b5076 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:00.828775 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2930e-42db-4f08-9bc0-194d6b2b5076 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:00.829758 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2930e-42db-4f08-9bc0-194d6b2b5076 None admin] RBAC: Adding query filter params (role.id=d1109db57b004dd9a4f84a634ff76205, scope.project.id=f81f2060d8dd4c8a85edacc1547c6b3c, user.id=4cf56ce5f5694a0499199ccd4ee6ab75) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:00.830081 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2930e-42db-4f08-9bc0-194d6b2b5076 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:00.831530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2930e-42db-4f08-9bc0-194d6b2b5076 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:00.844388 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 86/168] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:19:00 2022] GET /identity/v3/role_assignments?user.id=4cf56ce5f5694a0499199ccd4ee6ab75&scope.project.id=f81f2060d8dd4c8a85edacc1547c6b3c&role.id=d1109db57b004dd9a4f84a634ff76205 => generated 608 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:00.845294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2930e-42db-4f08-9bc0-194d6b2b5076 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:00.899440 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e77f3-78a7-45fa-9185-ee7d436ad3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:00.900187 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e77f3-78a7-45fa-9185-ee7d436ad3af None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:00.900473 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e77f3-78a7-45fa-9185-ee7d436ad3af None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:00.900750 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e77f3-78a7-45fa-9185-ee7d436ad3af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:00.926468 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:01.021703 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2930e-42db-4f08-9bc0-194d6b2b5076 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:01.046723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:01.082815 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf42453-a155-473d-9c23-450ec10d5cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:01.083477 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf42453-a155-473d-9c23-450ec10d5cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:01.083755 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf42453-a155-473d-9c23-450ec10d5cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:01.084027 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf42453-a155-473d-9c23-450ec10d5cf6 None admin] PATH_INFO: `/v3/users/4cf56ce5f5694a0499199ccd4ee6ab75` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:01.085261 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf42453-a155-473d-9c23-450ec10d5cf6 None admin] RBAC: Authorizing `identity:get_user(user_id=4cf56ce5f5694a0499199ccd4ee6ab75)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:01.086747 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf42453-a155-473d-9c23-450ec10d5cf6 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:01.088387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 87/169] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:19:00 2022] GET /identity/v3/users/4cf56ce5f5694a0499199ccd4ee6ab75 => generated 275 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:01.089264 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf42453-a155-473d-9c23-450ec10d5cf6 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:01.089380 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e77f3-78a7-45fa-9185-ee7d436ad3af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:01.090986 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e77f3-78a7-45fa-9185-ee7d436ad3af None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:01.275665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf42453-a155-473d-9c23-450ec10d5cf6 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:01.289031 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:01.296222 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 83/170] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:19:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 712 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 10:19:01.297095 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e77f3-78a7-45fa-9185-ee7d436ad3af None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:01.357740 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b32910-97c5-4b75-90da-8c7a98acb709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:01.358397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b32910-97c5-4b75-90da-8c7a98acb709 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:01.358683 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b32910-97c5-4b75-90da-8c7a98acb709 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:01.358959 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b32910-97c5-4b75-90da-8c7a98acb709 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:01.359631 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b32910-97c5-4b75-90da-8c7a98acb709 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:01.359938 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b32910-97c5-4b75-90da-8c7a98acb709 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:01.363172 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b32910-97c5-4b75-90da-8c7a98acb709 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:01.381541 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 88/171] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:19:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:01.382851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d6647-1df0-42fa-8dac-47eb394ad101 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:01.383135 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d6647-1df0-42fa-8dac-47eb394ad101 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:01.383412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d6647-1df0-42fa-8dac-47eb394ad101 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:01.387554 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 89/172] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:01.388857 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2d6647-1df0-42fa-8dac-47eb394ad101 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:01.442410 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:01.452920 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e77f3-78a7-45fa-9185-ee7d436ad3af None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:01.527275 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b5ec2-ad52-41b2-af5d-ce77bb85a08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:01.527895 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b5ec2-ad52-41b2-af5d-ce77bb85a08f None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:01.528171 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b5ec2-ad52-41b2-af5d-ce77bb85a08f None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:01.528445 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b5ec2-ad52-41b2-af5d-ce77bb85a08f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:01.606213 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2d6647-1df0-42fa-8dac-47eb394ad101 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:01.649696 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635b5ec2-ad52-41b2-af5d-ce77bb85a08f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:01.651253 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635b5ec2-ad52-41b2-af5d-ce77bb85a08f None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:01.654139 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42e982-bfc1-427c-aa51-ae70d3a0e838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:01.654876 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42e982-bfc1-427c-aa51-ae70d3a0e838 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:01.655218 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42e982-bfc1-427c-aa51-ae70d3a0e838 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:01.655654 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42e982-bfc1-427c-aa51-ae70d3a0e838 None admin] PATH_INFO: `/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:01.656977 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c42e982-bfc1-427c-aa51-ae70d3a0e838 None admin] RBAC: Authorizing `identity:get_project(project_id=f81f2060d8dd4c8a85edacc1547c6b3c)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:01.658590 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c42e982-bfc1-427c-aa51-ae70d3a0e838 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:01.660423 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 90/173] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:19:01 2022] GET /identity/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c => generated 298 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:01.662868 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42e982-bfc1-427c-aa51-ae70d3a0e838 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:01.711401 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:01.755148 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:01.774436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 84/174] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:19:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 10:19:01.776277 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053736b4-fb91-4fda-8af7-46087b4fb466 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:01.776635 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053736b4-fb91-4fda-8af7-46087b4fb466 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:01.776975 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053736b4-fb91-4fda-8af7-46087b4fb466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:01.786862 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42e982-bfc1-427c-aa51-ae70d3a0e838 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:01.802872 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-053736b4-fb91-4fda-8af7-46087b4fb466 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:01.804790 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-053736b4-fb91-4fda-8af7-46087b4fb466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:01.828128 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884e90f-43b4-49e1-98af-9813762cfb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:01.828744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884e90f-43b4-49e1-98af-9813762cfb24 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:01.829022 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884e90f-43b4-49e1-98af-9813762cfb24 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:01.829546 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884e90f-43b4-49e1-98af-9813762cfb24 None admin] PATH_INFO: `/v3/users/7ecff1fbd79248b7bcd1e69035be3515` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:01.830692 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884e90f-43b4-49e1-98af-9813762cfb24 None admin] RBAC: Authorizing `identity:get_user(user_id=7ecff1fbd79248b7bcd1e69035be3515)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:01.832128 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884e90f-43b4-49e1-98af-9813762cfb24 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:01.833714 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 91/175] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:19:01 2022] GET /identity/v3/users/7ecff1fbd79248b7bcd1e69035be3515 => generated 247 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:01.835194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac092bb-a63b-4770-af9f-d07d260fa637 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:01.835307 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac092bb-a63b-4770-af9f-d07d260fa637 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:01.835574 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac092bb-a63b-4770-af9f-d07d260fa637 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:01.838703 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 92/176] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:01.839587 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac092bb-a63b-4770-af9f-d07d260fa637 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:01.881972 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 85/177] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:01.882410 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053736b4-fb91-4fda-8af7-46087b4fb466 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:01.957393 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac092bb-a63b-4770-af9f-d07d260fa637 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:01.967259 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:01.987045 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053736b4-fb91-4fda-8af7-46087b4fb466 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:01.995222 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6542a-893c-4e01-b402-909724361874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:01.995831 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6542a-893c-4e01-b402-909724361874 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:01.996111 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6542a-893c-4e01-b402-909724361874 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:01.996385 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6542a-893c-4e01-b402-909724361874 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:01.997109 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a6542a-893c-4e01-b402-909724361874 None admin] RBAC: Adding query filter params (role.id=dc7cfc688d354df4bc870fe8621c48bd, scope.project.id=f81f2060d8dd4c8a85edacc1547c6b3c, user.id=4cf56ce5f5694a0499199ccd4ee6ab75) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:01.997938 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a6542a-893c-4e01-b402-909724361874 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:01.999486 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a6542a-893c-4e01-b402-909724361874 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:02.010287 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 93/178] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:19:01 2022] GET /identity/v3/role_assignments?user.id=4cf56ce5f5694a0499199ccd4ee6ab75&scope.project.id=f81f2060d8dd4c8a85edacc1547c6b3c&role.id=dc7cfc688d354df4bc870fe8621c48bd => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:02.010287 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee3193-0415-4bd2-80d7-0a509edee7f8 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:02.010287 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee3193-0415-4bd2-80d7-0a509edee7f8 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:02.010287 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee3193-0415-4bd2-80d7-0a509edee7f8 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:02.010287 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 94/179] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:02.010287 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee3193-0415-4bd2-80d7-0a509edee7f8 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:02.011089 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:02.032960 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc95b4-7e65-4e0a-82dc-ec61fd622796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:02.033907 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc95b4-7e65-4e0a-82dc-ec61fd622796 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:02.034353 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc95b4-7e65-4e0a-82dc-ec61fd622796 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:02.034796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc95b4-7e65-4e0a-82dc-ec61fd622796 None admin] PATH_INFO: `/v3/projects/51fc4712d2454146b6103cf60db3fb7a` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:02.036302 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dc95b4-7e65-4e0a-82dc-ec61fd622796 None admin] RBAC: Authorizing `identity:get_project(project_id=51fc4712d2454146b6103cf60db3fb7a)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:02.038219 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dc95b4-7e65-4e0a-82dc-ec61fd622796 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:02.040676 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 86/180] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:19:01 2022] GET /identity/v3/projects/51fc4712d2454146b6103cf60db3fb7a => generated 312 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:02.041907 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77326320-c659-4733-8e00-c0915c2851fc None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:02.042210 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77326320-c659-4733-8e00-c0915c2851fc None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:02.042482 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77326320-c659-4733-8e00-c0915c2851fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:02.063138 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-77326320-c659-4733-8e00-c0915c2851fc None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:02.064368 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-77326320-c659-4733-8e00-c0915c2851fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:02.145899 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 87/181] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:02.147236 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d51314-f3b8-4be3-a2e4-5a932e2a19ea None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:02.147523 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d51314-f3b8-4be3-a2e4-5a932e2a19ea None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:02.147796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d51314-f3b8-4be3-a2e4-5a932e2a19ea None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:02.151277 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee3193-0415-4bd2-80d7-0a509edee7f8 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:02.153457 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 88/182] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:02.154909 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c3a46-675a-46b2-b3c9-616b947c3007 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:02.155193 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c3a46-675a-46b2-b3c9-616b947c3007 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:02.155471 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c3a46-675a-46b2-b3c9-616b947c3007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:02.176132 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-473c3a46-675a-46b2-b3c9-616b947c3007 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:02.177420 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-473c3a46-675a-46b2-b3c9-616b947c3007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:02.194541 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c6df6-52b0-4606-8569-a29b2e7600e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:02.196094 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c6df6-52b0-4606-8569-a29b2e7600e5 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:02.196862 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c6df6-52b0-4606-8569-a29b2e7600e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:02.197340 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c6df6-52b0-4606-8569-a29b2e7600e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:02.258659 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:02.265017 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 89/183] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:02.265164 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c3a46-675a-46b2-b3c9-616b947c3007 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:02.289431 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35c6df6-52b0-4606-8569-a29b2e7600e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:02.291259 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35c6df6-52b0-4606-8569-a29b2e7600e5 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:02.323639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:02.393754 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c3a46-675a-46b2-b3c9-616b947c3007 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:02.410603 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 95/184] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:19:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 10:19:02.412170 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c9c79-4466-46b3-889f-68de0a431848 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:02.412781 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c9c79-4466-46b3-889f-68de0a431848 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:02.413261 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c9c79-4466-46b3-889f-68de0a431848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:02.432901 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-bf3c9c79-4466-46b3-889f-68de0a431848 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:02.434750 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-bf3c9c79-4466-46b3-889f-68de0a431848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:02.435998 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1200f-40f2-4bc8-bf28-4ca377830b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:02.436462 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1200f-40f2-4bc8-bf28-4ca377830b28 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:02.436741 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1200f-40f2-4bc8-bf28-4ca377830b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:02.437012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1200f-40f2-4bc8-bf28-4ca377830b28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:02.437742 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb1200f-40f2-4bc8-bf28-4ca377830b28 None admin] RBAC: Adding query filter params (role.id=dc7cfc688d354df4bc870fe8621c48bd, scope.project.id=51fc4712d2454146b6103cf60db3fb7a, user.id=4cf56ce5f5694a0499199ccd4ee6ab75) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:02.438066 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb1200f-40f2-4bc8-bf28-4ca377830b28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:02.440242 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb1200f-40f2-4bc8-bf28-4ca377830b28 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:02.445658 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 90/185] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:19:02 2022] GET /identity/v3/role_assignments?user.id=4cf56ce5f5694a0499199ccd4ee6ab75&scope.project.id=51fc4712d2454146b6103cf60db3fb7a&role.id=dc7cfc688d354df4bc870fe8621c48bd => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:02.446493 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1200f-40f2-4bc8-bf28-4ca377830b28 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:02.530956 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 96/186] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:02.531671 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c9c79-4466-46b3-889f-68de0a431848 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:02.546609 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:02.576791 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1200f-40f2-4bc8-bf28-4ca377830b28 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:02.588746 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:02.618877 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c0a3e-1d83-4bfc-bb5b-d138226710d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:02.619531 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c0a3e-1d83-4bfc-bb5b-d138226710d7 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:02.619808 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c0a3e-1d83-4bfc-bb5b-d138226710d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:02.620081 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c0a3e-1d83-4bfc-bb5b-d138226710d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:02.633247 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c0a3e-1d83-4bfc-bb5b-d138226710d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:02.639606 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-a23c0a3e-1d83-4bfc-bb5b-d138226710d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:19:02.639606 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:02.639606 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:02.639606 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:02.639606 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:02.639606 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:02.639606 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:02.639606 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:02.639606 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:02.639606 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:02.639606 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:02.639606 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:02.639606 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:02.639606 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:02.639606 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:02.639606 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:02.640543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:02.640543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:02.640543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:02.640543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:02.640543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:02.640543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:02.640543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:02.640543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:02.640543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:02.640543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:02.640543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:02.640543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:02.640543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:02.640543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:02.640543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:02.640543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:02.641326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:02.641326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:02.641326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:02.641326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:02.641326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:02.641326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:02.641326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:02.641326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:02.641326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:02.641326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:02.641326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:19:02.641326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:02.641326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 91/187] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:19:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:02.641989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c0a3e-1d83-4bfc-bb5b-d138226710d7 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:02.680863 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c9c79-4466-46b3-889f-68de0a431848 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:02.733965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aac084-d9b4-4bfb-a5eb-046773d2f252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:02.734655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aac084-d9b4-4bfb-a5eb-046773d2f252 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:02.735327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aac084-d9b4-4bfb-a5eb-046773d2f252 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:02.735327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aac084-d9b4-4bfb-a5eb-046773d2f252 None admin] PATH_INFO: `/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:02.736893 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aac084-d9b4-4bfb-a5eb-046773d2f252 None admin] RBAC: Authorizing `identity:get_project(project_id=f81f2060d8dd4c8a85edacc1547c6b3c)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:02.738378 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aac084-d9b4-4bfb-a5eb-046773d2f252 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:02.740217 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 97/188] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:19:02 2022] GET /identity/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:02.740810 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aac084-d9b4-4bfb-a5eb-046773d2f252 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:02.779218 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c0a3e-1d83-4bfc-bb5b-d138226710d7 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:02.817543 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:02.825131 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc87773-4065-4d54-a7ed-30c62137eabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:02.826336 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc87773-4065-4d54-a7ed-30c62137eabb None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:02.826619 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc87773-4065-4d54-a7ed-30c62137eabb None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:02.826892 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc87773-4065-4d54-a7ed-30c62137eabb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:02.839917 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc87773-4065-4d54-a7ed-30c62137eabb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:02.845775 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-edc87773-4065-4d54-a7ed-30c62137eabb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 10:19:02.845775 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:02.845775 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:02.845775 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:02.845775 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:02.845775 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:02.845775 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:02.845775 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:02.845775 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:02.845775 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:02.845775 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:02.845775 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:02.845775 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:02.845775 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:02.845775 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:02.845775 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:02.846806 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:02.846806 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:02.846806 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:02.846806 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:02.846806 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:02.846806 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:02.846806 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:02.846806 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:02.846806 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:02.846806 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:02.846806 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:02.846806 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:02.846806 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:02.846806 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:02.846806 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:02.846806 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:02.847618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:02.847618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:02.847618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:02.847618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:02.847618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:02.847618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:02.847618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:02.847618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:02.847618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:02.847618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:02.847618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 10:19:02.847618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:02.847618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 92/189] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 10:19:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:02.848879 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc87773-4065-4d54-a7ed-30c62137eabb None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:02.859285 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aac084-d9b4-4bfb-a5eb-046773d2f252 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:02.880142 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:02.895598 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95a825-6bc8-4471-afaa-68eb2f54eecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:02.896236 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95a825-6bc8-4471-afaa-68eb2f54eecb None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:02.896516 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95a825-6bc8-4471-afaa-68eb2f54eecb None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:02.896797 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95a825-6bc8-4471-afaa-68eb2f54eecb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:02.906906 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e95a825-6bc8-4471-afaa-68eb2f54eecb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:02.912178 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-2e95a825-6bc8-4471-afaa-68eb2f54eecb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 10:19:02.912178 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:02.912178 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:02.912178 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:02.912178 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:02.912178 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:02.912178 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:02.912178 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:02.912178 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:02.912178 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:02.912178 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:02.912178 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:02.912178 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:02.912178 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:02.912178 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:02.912178 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:02.913209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:02.913209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:02.913209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:02.913209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:02.913209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:02.913209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:02.913209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:02.913209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:02.913209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:02.913209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:02.913209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:02.913209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:02.913209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:02.913209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:02.913209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:02.913209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:02.914195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:02.914195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:02.914195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:02.914195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:02.914195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:02.914195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:02.914195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:02.914195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:02.914195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:02.914195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:02.914195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 10:19:02.914195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:02.914195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 98/190] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:19:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:02.914195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95a825-6bc8-4471-afaa-68eb2f54eecb None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:02.957447 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc87773-4065-4d54-a7ed-30c62137eabb None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:02.989019 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29404e34-c3f9-48aa-8e13-70935198ebae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:02.989616 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29404e34-c3f9-48aa-8e13-70935198ebae None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:02.989889 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29404e34-c3f9-48aa-8e13-70935198ebae None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:02.990189 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29404e34-c3f9-48aa-8e13-70935198ebae None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:02.990836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29404e34-c3f9-48aa-8e13-70935198ebae None admin] RBAC: Adding query filter params (name=admin) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:02.991130 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29404e34-c3f9-48aa-8e13-70935198ebae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:02.992496 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29404e34-c3f9-48aa-8e13-70935198ebae None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:03.000911 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 93/191] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:19:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:03.001695 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29404e34-c3f9-48aa-8e13-70935198ebae None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:03.019857 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95a825-6bc8-4471-afaa-68eb2f54eecb None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:03.049267 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:03.055774 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c65d25-021c-4213-acfc-659353f648b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:03.056316 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c65d25-021c-4213-acfc-659353f648b4 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:03.056586 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c65d25-021c-4213-acfc-659353f648b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:03.056853 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c65d25-021c-4213-acfc-659353f648b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:03.057565 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c65d25-021c-4213-acfc-659353f648b4 None admin] RBAC: Adding query filter params (role.id=532b37d9076c45618cf352aafb85f35e, scope.project.id=f81f2060d8dd4c8a85edacc1547c6b3c, user.id=7ecff1fbd79248b7bcd1e69035be3515) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:03.057863 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c65d25-021c-4213-acfc-659353f648b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:03.059328 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c65d25-021c-4213-acfc-659353f648b4 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:03.065468 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 99/192] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:19:02 2022] GET /identity/v3/role_assignments?user.id=7ecff1fbd79248b7bcd1e69035be3515&scope.project.id=f81f2060d8dd4c8a85edacc1547c6b3c&role.id=532b37d9076c45618cf352aafb85f35e => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:03.066935 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c65d25-021c-4213-acfc-659353f648b4 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:03.099299 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:03.111758 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29404e34-c3f9-48aa-8e13-70935198ebae None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:03.156414 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c43556-f1c5-4b12-aaf3-f536c27cf237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:03.157063 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c43556-f1c5-4b12-aaf3-f536c27cf237 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:03.157341 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c43556-f1c5-4b12-aaf3-f536c27cf237 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:03.157611 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c43556-f1c5-4b12-aaf3-f536c27cf237 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:03.158839 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c43556-f1c5-4b12-aaf3-f536c27cf237 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:03.160355 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c43556-f1c5-4b12-aaf3-f536c27cf237 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:03.162251 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 94/193] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:03.162922 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c43556-f1c5-4b12-aaf3-f536c27cf237 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:03.192217 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c65d25-021c-4213-acfc-659353f648b4 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:03.229242 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb888de-67e4-4ab0-8e65-601b1ab21fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:03.229864 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb888de-67e4-4ab0-8e65-601b1ab21fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:03.230202 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb888de-67e4-4ab0-8e65-601b1ab21fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:03.230479 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb888de-67e4-4ab0-8e65-601b1ab21fe0 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:03.231152 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb888de-67e4-4ab0-8e65-601b1ab21fe0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:03.231451 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb888de-67e4-4ab0-8e65-601b1ab21fe0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:03.233395 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb888de-67e4-4ab0-8e65-601b1ab21fe0 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:03.242341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 100/194] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:19:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:03.243048 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb888de-67e4-4ab0-8e65-601b1ab21fe0 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:03.275366 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c43556-f1c5-4b12-aaf3-f536c27cf237 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:03.314928 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9bda38-db50-4a8d-9aa3-f1818cbd3265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:03.315462 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9bda38-db50-4a8d-9aa3-f1818cbd3265 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:03.315731 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9bda38-db50-4a8d-9aa3-f1818cbd3265 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:03.315999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9bda38-db50-4a8d-9aa3-f1818cbd3265 None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:03.316328 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:03.316656 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9bda38-db50-4a8d-9aa3-f1818cbd3265 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:03.317373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9bda38-db50-4a8d-9aa3-f1818cbd3265 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:03.318795 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9bda38-db50-4a8d-9aa3-f1818cbd3265 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:03.330301 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 95/195] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 10:19:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:03.334320 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9bda38-db50-4a8d-9aa3-f1818cbd3265 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:03.356356 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:03.364171 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb888de-67e4-4ab0-8e65-601b1ab21fe0 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:03.399386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b5a7f-d14b-4dbf-b086-38e0812bcd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:03.400033 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b5a7f-d14b-4dbf-b086-38e0812bcd09 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:03.400309 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b5a7f-d14b-4dbf-b086-38e0812bcd09 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:03.400580 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b5a7f-d14b-4dbf-b086-38e0812bcd09 None admin] PATH_INFO: `/v3/users/b6f92cb9fa9a479e88be642cebb82831` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:03.416308 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838b5a7f-d14b-4dbf-b086-38e0812bcd09 None admin] RBAC: Authorizing `identity:get_user(user_id=b6f92cb9fa9a479e88be642cebb82831)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:03.417745 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838b5a7f-d14b-4dbf-b086-38e0812bcd09 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:03.419416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 101/196] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:19:03 2022] GET /identity/v3/users/b6f92cb9fa9a479e88be642cebb82831 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:03.420168 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b5a7f-d14b-4dbf-b086-38e0812bcd09 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:03.437270 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9bda38-db50-4a8d-9aa3-f1818cbd3265 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:03.471171 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0903800a-1df2-412f-9ce3-8dc4762e07a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:03.471774 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0903800a-1df2-412f-9ce3-8dc4762e07a3 None admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:03.472049 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0903800a-1df2-412f-9ce3-8dc4762e07a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:03.472317 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0903800a-1df2-412f-9ce3-8dc4762e07a3 None admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:03.472961 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0903800a-1df2-412f-9ce3-8dc4762e07a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:03.474416 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0903800a-1df2-412f-9ce3-8dc4762e07a3 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:03.495247 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 96/197] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 10:19:03 2022] POST /identity/v3/users => generated 297 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:19:03.495247 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0903800a-1df2-412f-9ce3-8dc4762e07a3 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:03.531945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b5a7f-d14b-4dbf-b086-38e0812bcd09 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:03.548208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:03.571843 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9af050-bede-4eee-975a-7e51e3dbbd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:03.572407 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9af050-bede-4eee-975a-7e51e3dbbd0b None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:03.572673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9af050-bede-4eee-975a-7e51e3dbbd0b None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:03.572935 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9af050-bede-4eee-975a-7e51e3dbbd0b None admin] PATH_INFO: `/v3/users/b6f92cb9fa9a479e88be642cebb82831` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:03.574156 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9af050-bede-4eee-975a-7e51e3dbbd0b None admin] RBAC: Authorizing `identity:get_user(user_id=b6f92cb9fa9a479e88be642cebb82831)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:03.576296 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9af050-bede-4eee-975a-7e51e3dbbd0b None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:03.582745 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 102/198] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:19:03 2022] GET /identity/v3/users/b6f92cb9fa9a479e88be642cebb82831 => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:03.582745 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9af050-bede-4eee-975a-7e51e3dbbd0b None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:03.603188 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:03.631572 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0903800a-1df2-412f-9ce3-8dc4762e07a3 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:03.667415 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34581da5-2599-4a4e-a92d-0e08268faef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:03.667820 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34581da5-2599-4a4e-a92d-0e08268faef9 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:03.668275 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34581da5-2599-4a4e-a92d-0e08268faef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:03.668382 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34581da5-2599-4a4e-a92d-0e08268faef9 None admin] PATH_INFO: `/v3/users/9b1ac2894c834148b68b48b22b199c2d` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:03.670470 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34581da5-2599-4a4e-a92d-0e08268faef9 None admin] RBAC: Authorizing `identity:get_user(user_id=9b1ac2894c834148b68b48b22b199c2d)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:03.671926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34581da5-2599-4a4e-a92d-0e08268faef9 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:03.673721 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 97/199] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:19:03 2022] GET /identity/v3/users/9b1ac2894c834148b68b48b22b199c2d => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:03.674328 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34581da5-2599-4a4e-a92d-0e08268faef9 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:03.699386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9af050-bede-4eee-975a-7e51e3dbbd0b None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:03.744986 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb003c-278f-4638-b6d2-36f761bd9543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:03.745608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb003c-278f-4638-b6d2-36f761bd9543 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:03.745886 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb003c-278f-4638-b6d2-36f761bd9543 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:03.746189 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb003c-278f-4638-b6d2-36f761bd9543 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:03.809209 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34581da5-2599-4a4e-a92d-0e08268faef9 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:03.822466 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:03.846841 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb003c-278f-4638-b6d2-36f761bd9543 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:03.848249 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dcb275-ea53-4c60-b882-d173fcabcecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:03.848372 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb003c-278f-4638-b6d2-36f761bd9543 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:03.848857 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dcb275-ea53-4c60-b882-d173fcabcecc None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:03.849130 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dcb275-ea53-4c60-b882-d173fcabcecc None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:03.849396 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dcb275-ea53-4c60-b882-d173fcabcecc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:03.870684 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:03.930860 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dcb275-ea53-4c60-b882-d173fcabcecc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:03.932618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dcb275-ea53-4c60-b882-d173fcabcecc None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:03.966169 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 103/200] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:19:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 10:19:03.967165 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb003c-278f-4638-b6d2-36f761bd9543 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:04.038139 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 98/201] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:19:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 10:19:04.039082 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dcb275-ea53-4c60-b882-d173fcabcecc None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:04.066694 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:04.087521 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb003c-278f-4638-b6d2-36f761bd9543 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:04.090932 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:04.130111 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96ba03-7d16-434a-a960-56867d7b2b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:04.130728 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96ba03-7d16-434a-a960-56867d7b2b92 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:04.131006 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96ba03-7d16-434a-a960-56867d7b2b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:04.131281 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96ba03-7d16-434a-a960-56867d7b2b92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:04.151731 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dcb275-ea53-4c60-b882-d173fcabcecc None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:04.190351 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454cb906-451d-4123-8192-755d4d30898e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:04.191299 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cb906-451d-4123-8192-755d4d30898e None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:04.191766 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cb906-451d-4123-8192-755d4d30898e None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:04.192211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cb906-451d-4123-8192-755d4d30898e None admin] PATH_INFO: `/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:04.193525 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454cb906-451d-4123-8192-755d4d30898e None admin] RBAC: Authorizing `identity:get_project(project_id=c3a5c629197a4f8ba3a3d1ad27faaa7e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:04.195266 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454cb906-451d-4123-8192-755d4d30898e None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:04.197094 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 99/202] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:19:04 2022] GET /identity/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:04.198126 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454cb906-451d-4123-8192-755d4d30898e None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:04.225390 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a96ba03-7d16-434a-a960-56867d7b2b92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:04.227727 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a96ba03-7d16-434a-a960-56867d7b2b92 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:04.298172 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454cb906-451d-4123-8192-755d4d30898e None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:04.318357 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:04.331330 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:04.340964 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a17404-3f9e-4be4-a2ed-0c8394295892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:04.342771 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a17404-3f9e-4be4-a2ed-0c8394295892 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:04.343511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a17404-3f9e-4be4-a2ed-0c8394295892 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:04.343985 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a17404-3f9e-4be4-a2ed-0c8394295892 None admin] PATH_INFO: `/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:04.345933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a17404-3f9e-4be4-a2ed-0c8394295892 None admin] RBAC: Authorizing `identity:get_project(project_id=c3a5c629197a4f8ba3a3d1ad27faaa7e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:04.348663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a17404-3f9e-4be4-a2ed-0c8394295892 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:04.351325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 100/203] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:19:04 2022] GET /identity/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:04.352308 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a17404-3f9e-4be4-a2ed-0c8394295892 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:04.364538 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 104/204] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:19:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 10:19:04.365515 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96ba03-7d16-434a-a960-56867d7b2b92 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:04.474731 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a17404-3f9e-4be4-a2ed-0c8394295892 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:04.491731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96ba03-7d16-434a-a960-56867d7b2b92 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:04.509981 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970aee26-a51c-4a24-b601-36ef5dd3c490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:04.510639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970aee26-a51c-4a24-b601-36ef5dd3c490 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:04.510938 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970aee26-a51c-4a24-b601-36ef5dd3c490 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:04.511209 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970aee26-a51c-4a24-b601-36ef5dd3c490 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:04.511986 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970aee26-a51c-4a24-b601-36ef5dd3c490 None admin] RBAC: Adding query filter params (role.id=532b37d9076c45618cf352aafb85f35e, scope.project.id=c3a5c629197a4f8ba3a3d1ad27faaa7e, user.id=b6f92cb9fa9a479e88be642cebb82831) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:04.512560 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970aee26-a51c-4a24-b601-36ef5dd3c490 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:04.514368 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970aee26-a51c-4a24-b601-36ef5dd3c490 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:04.520187 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 101/205] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:19:04 2022] GET /identity/v3/role_assignments?user.id=b6f92cb9fa9a479e88be642cebb82831&scope.project.id=c3a5c629197a4f8ba3a3d1ad27faaa7e&role.id=532b37d9076c45618cf352aafb85f35e => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:04.521136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970aee26-a51c-4a24-b601-36ef5dd3c490 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:04.541873 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd384b30-b575-4f28-8cd7-3efac4d909d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:04.543191 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd384b30-b575-4f28-8cd7-3efac4d909d6 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:04.544143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd384b30-b575-4f28-8cd7-3efac4d909d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:04.545150 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd384b30-b575-4f28-8cd7-3efac4d909d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:04.546158 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd384b30-b575-4f28-8cd7-3efac4d909d6 None admin] RBAC: Adding query filter params (role.id=d1109db57b004dd9a4f84a634ff76205, scope.project.id=c3a5c629197a4f8ba3a3d1ad27faaa7e, user.id=b6f92cb9fa9a479e88be642cebb82831) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:04.546743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd384b30-b575-4f28-8cd7-3efac4d909d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:04.549519 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd384b30-b575-4f28-8cd7-3efac4d909d6 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:04.558851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 105/206] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:19:04 2022] GET /identity/v3/role_assignments?user.id=b6f92cb9fa9a479e88be642cebb82831&scope.project.id=c3a5c629197a4f8ba3a3d1ad27faaa7e&role.id=d1109db57b004dd9a4f84a634ff76205 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:04.600228 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:04.685713 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970aee26-a51c-4a24-b601-36ef5dd3c490 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:04.726396 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1856ef25-718f-4c8c-b8a9-a1adb83fca61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:04.727061 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856ef25-718f-4c8c-b8a9-a1adb83fca61 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:04.727338 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856ef25-718f-4c8c-b8a9-a1adb83fca61 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:04.727609 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856ef25-718f-4c8c-b8a9-a1adb83fca61 None admin] PATH_INFO: `/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:04.728799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1856ef25-718f-4c8c-b8a9-a1adb83fca61 None admin] RBAC: Authorizing `identity:get_project(project_id=f81f2060d8dd4c8a85edacc1547c6b3c)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:04.730373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1856ef25-718f-4c8c-b8a9-a1adb83fca61 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:04.737223 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 102/207] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:19:04 2022] GET /identity/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:04.738793 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd384b30-b575-4f28-8cd7-3efac4d909d6 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:04.810053 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:04.878140 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd384b30-b575-4f28-8cd7-3efac4d909d6 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:04.917572 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030c169-cb96-4eb6-be3f-33953678b3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:04.918221 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030c169-cb96-4eb6-be3f-33953678b3d2 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:04.918504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030c169-cb96-4eb6-be3f-33953678b3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:04.918777 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030c169-cb96-4eb6-be3f-33953678b3d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:04.919560 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030c169-cb96-4eb6-be3f-33953678b3d2 None admin] RBAC: Adding query filter params (role.id=4ffa6e25be8941078cca7449c84e50d5, scope.project.id=f81f2060d8dd4c8a85edacc1547c6b3c, user.id=9b1ac2894c834148b68b48b22b199c2d) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:04.919869 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030c169-cb96-4eb6-be3f-33953678b3d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:04.921325 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030c169-cb96-4eb6-be3f-33953678b3d2 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:04.930691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 106/208] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:19:04 2022] GET /identity/v3/role_assignments?user.id=9b1ac2894c834148b68b48b22b199c2d&scope.project.id=f81f2060d8dd4c8a85edacc1547c6b3c&role.id=4ffa6e25be8941078cca7449c84e50d5 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:05.923859 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741a25e-d9d1-4712-95bd-3118c8c5fed4 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:05.924390 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741a25e-d9d1-4712-95bd-3118c8c5fed4 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:05.924390 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741a25e-d9d1-4712-95bd-3118c8c5fed4 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:05.925727 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 103/209] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:05.930850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e1b50-71bf-4d50-a6e4-7bfea54550e3 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:05.931246 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e1b50-71bf-4d50-a6e4-7bfea54550e3 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:05.931425 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e1b50-71bf-4d50-a6e4-7bfea54550e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:05.949918 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-e98e1b50-71bf-4d50-a6e4-7bfea54550e3 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:05.951170 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-e98e1b50-71bf-4d50-a6e4-7bfea54550e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:06.022047 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 107/210] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:06.189711 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776f1c2-0fac-4237-abdf-801c3f5bac08 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:06.190391 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776f1c2-0fac-4237-abdf-801c3f5bac08 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:06.190898 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776f1c2-0fac-4237-abdf-801c3f5bac08 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:06.192488 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 104/211] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:06.197810 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065567b5-5a12-4fa5-b27b-857f183df580 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:06.198113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065567b5-5a12-4fa5-b27b-857f183df580 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:06.198385 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065567b5-5a12-4fa5-b27b-857f183df580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:06.218233 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-065567b5-5a12-4fa5-b27b-857f183df580 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:06.219405 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-065567b5-5a12-4fa5-b27b-857f183df580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:06.255578 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:06.290644 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 108/212] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:06.293486 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776f1c2-0fac-4237-abdf-801c3f5bac08 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:06.408298 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776f1c2-0fac-4237-abdf-801c3f5bac08 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:06.423233 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:06.444779 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee6e64-f23f-4a7b-bbec-3d5fa98ad705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:06.445336 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee6e64-f23f-4a7b-bbec-3d5fa98ad705 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:06.445612 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee6e64-f23f-4a7b-bbec-3d5fa98ad705 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:06.445879 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee6e64-f23f-4a7b-bbec-3d5fa98ad705 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:06.457008 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee6e64-f23f-4a7b-bbec-3d5fa98ad705 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:06.462477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-ceee6e64-f23f-4a7b-bbec-3d5fa98ad705 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:19:06.462477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:06.462477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:06.462477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:06.462477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:06.462477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:06.462477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:06.462477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:06.462477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:06.462477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:06.462477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:06.462477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:06.462477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:06.462477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:06.462477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:06.462477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:06.463347 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:06.463347 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:06.463347 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:06.463347 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:06.463347 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:06.463347 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:06.463347 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:06.463347 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:06.463347 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:06.463347 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:06.463347 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:06.463347 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:06.463347 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:06.463347 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:06.463347 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:06.463347 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:06.464174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:06.464174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:06.464174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:06.464174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:06.464174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:06.464174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:06.464174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:06.464174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:06.464174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:06.464174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:06.464174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:19:06.464174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:06.464174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 105/213] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:19:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:06.469890 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065567b5-5a12-4fa5-b27b-857f183df580 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:06.570320 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee6e64-f23f-4a7b-bbec-3d5fa98ad705 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:06.572449 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065567b5-5a12-4fa5-b27b-857f183df580 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:06.609558 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1fa772-2848-442d-bb02-ec0a8f8d2ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:06.610620 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1fa772-2848-442d-bb02-ec0a8f8d2ded None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:06.611105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1fa772-2848-442d-bb02-ec0a8f8d2ded None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:06.611650 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1fa772-2848-442d-bb02-ec0a8f8d2ded None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:06.612558 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1fa772-2848-442d-bb02-ec0a8f8d2ded None admin] RBAC: Adding query filter params (name=member) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:06.613071 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1fa772-2848-442d-bb02-ec0a8f8d2ded None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:06.614693 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1fa772-2848-442d-bb02-ec0a8f8d2ded None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:06.623470 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 109/214] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 10:19:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:06.629574 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1fa772-2848-442d-bb02-ec0a8f8d2ded None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:06.674897 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:06.698151 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee6e64-f23f-4a7b-bbec-3d5fa98ad705 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:06.743795 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692cef45-b691-4074-844b-31abf65121df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:06.744709 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cef45-b691-4074-844b-31abf65121df None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:06.745050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cef45-b691-4074-844b-31abf65121df None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:06.745395 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cef45-b691-4074-844b-31abf65121df None admin] PATH_INFO: `/v3/domains/default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:06.747544 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692cef45-b691-4074-844b-31abf65121df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:06.749336 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692cef45-b691-4074-844b-31abf65121df None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:06.751933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 106/215] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:06.757893 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692cef45-b691-4074-844b-31abf65121df None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:06.760532 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1fa772-2848-442d-bb02-ec0a8f8d2ded None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:06.809956 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09177a7f-5124-4cc5-bd2c-3e1ac9a881ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:06.810884 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09177a7f-5124-4cc5-bd2c-3e1ac9a881ad None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:06.811473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09177a7f-5124-4cc5-bd2c-3e1ac9a881ad None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:06.811950 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09177a7f-5124-4cc5-bd2c-3e1ac9a881ad None admin] PATH_INFO: `/v3/users/92cdd324cc6545ae9e3f84678c6b4254` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:06.813752 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:06.828641 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09177a7f-5124-4cc5-bd2c-3e1ac9a881ad None admin] RBAC: Authorizing `identity:get_user(user_id=92cdd324cc6545ae9e3f84678c6b4254)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:06.830823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09177a7f-5124-4cc5-bd2c-3e1ac9a881ad None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:06.833146 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 110/216] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:19:06 2022] GET /identity/v3/users/92cdd324cc6545ae9e3f84678c6b4254 => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:06.839443 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09177a7f-5124-4cc5-bd2c-3e1ac9a881ad None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:06.886481 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692cef45-b691-4074-844b-31abf65121df None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:06.922044 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58a142e-7898-4b78-bba5-bcb1b9123ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:06.922657 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a142e-7898-4b78-bba5-bcb1b9123ea1 None admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:06.922933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a142e-7898-4b78-bba5-bcb1b9123ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:06.923204 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a142e-7898-4b78-bba5-bcb1b9123ea1 None admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:06.923882 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58a142e-7898-4b78-bba5-bcb1b9123ea1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:06.925317 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58a142e-7898-4b78-bba5-bcb1b9123ea1 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:06.945016 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:06.947449 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 107/217] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 10:19:06 2022] POST /identity/v3/users => generated 297 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:19:06.948870 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc2c01-2b47-498e-99de-c50d64ff5327 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:06.949167 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc2c01-2b47-498e-99de-c50d64ff5327 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:06.949522 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc2c01-2b47-498e-99de-c50d64ff5327 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:06.951618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 108/218] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:06.957565 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1acec9-40da-44f8-bcd5-e06b904545e5 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:06.957924 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1acec9-40da-44f8-bcd5-e06b904545e5 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:06.958301 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1acec9-40da-44f8-bcd5-e06b904545e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:06.967997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09177a7f-5124-4cc5-bd2c-3e1ac9a881ad None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:06.978000 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-cb1acec9-40da-44f8-bcd5-e06b904545e5 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:06.979289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-cb1acec9-40da-44f8-bcd5-e06b904545e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:07.007383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de2694-1d51-494a-9f95-e5848b243f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:07.008019 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de2694-1d51-494a-9f95-e5848b243f10 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:07.008304 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de2694-1d51-494a-9f95-e5848b243f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:07.008974 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de2694-1d51-494a-9f95-e5848b243f10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:07.059129 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 109/219] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 10:19:07.066383 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b33c588-668b-44ed-9331-ff1c4e96a677 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:07.066383 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b33c588-668b-44ed-9331-ff1c4e96a677 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:07.066383 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b33c588-668b-44ed-9331-ff1c4e96a677 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:07.066383 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 110/220] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:07.068946 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea4524-d145-4c47-8762-fcd83bd574b8 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:07.068946 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea4524-d145-4c47-8762-fcd83bd574b8 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:07.068946 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea4524-d145-4c47-8762-fcd83bd574b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:07.088680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-53ea4524-d145-4c47-8762-fcd83bd574b8 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:07.090925 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-53ea4524-d145-4c47-8762-fcd83bd574b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:07.111552 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de2694-1d51-494a-9f95-e5848b243f10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:07.113643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de2694-1d51-494a-9f95-e5848b243f10 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:07.124580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:07.193167 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 111/221] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 10:19:07.232052 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:07.243378 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 111/222] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:19:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 10:19:07.249653 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea4524-d145-4c47-8762-fcd83bd574b8 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:07.371373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea4524-d145-4c47-8762-fcd83bd574b8 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:07.371581 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de2694-1d51-494a-9f95-e5848b243f10 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:07.416590 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be73d9be-2127-4b0c-bf29-7504553b42cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:07.417232 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73d9be-2127-4b0c-bf29-7504553b42cc None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:07.417505 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73d9be-2127-4b0c-bf29-7504553b42cc None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:07.417780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73d9be-2127-4b0c-bf29-7504553b42cc None admin] PATH_INFO: `/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:07.419529 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be73d9be-2127-4b0c-bf29-7504553b42cc None admin] RBAC: Authorizing `identity:get_project(project_id=c3a5c629197a4f8ba3a3d1ad27faaa7e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:07.420426 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be73d9be-2127-4b0c-bf29-7504553b42cc None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:07.426117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 112/223] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:19:07 2022] GET /identity/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:07.435856 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be73d9be-2127-4b0c-bf29-7504553b42cc None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:07.476560 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:07.517482 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de2694-1d51-494a-9f95-e5848b243f10 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:07.557752 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114e451-4729-4c1d-b7c7-3998d2c77a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:07.558440 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114e451-4729-4c1d-b7c7-3998d2c77a06 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:07.558720 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114e451-4729-4c1d-b7c7-3998d2c77a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:07.558994 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114e451-4729-4c1d-b7c7-3998d2c77a06 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:07.559186 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be73d9be-2127-4b0c-bf29-7504553b42cc None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:07.570927 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114e451-4729-4c1d-b7c7-3998d2c77a06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:07.576055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-4114e451-4729-4c1d-b7c7-3998d2c77a06 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 10:19:07.576055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:07.576055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:07.576055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:07.576055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:07.576055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:07.576055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:07.576055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:07.576055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:07.576055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:07.576055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:07.576055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:07.576055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:07.576055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:07.576055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:07.576055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:07.576947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:07.576947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:07.576947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:07.576947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:07.576947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:07.576947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:07.576947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:07.576947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:07.576947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:07.576947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:07.576947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:07.576947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:07.576947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:07.576947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:07.576947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:07.576947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:07.577730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:07.577730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:07.577730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:07.577730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:07.577730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:07.577730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:07.577730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:07.577730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:07.577730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:07.577730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:07.577730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 10:19:07.577730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:07.577730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 112/224] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 10:19:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:19:07.578889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114e451-4729-4c1d-b7c7-3998d2c77a06 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:07.602532 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f285eb52-7f88-474d-bdec-a81890176513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:07.603215 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285eb52-7f88-474d-bdec-a81890176513 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:07.604006 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285eb52-7f88-474d-bdec-a81890176513 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:07.604329 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285eb52-7f88-474d-bdec-a81890176513 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:07.605339 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f285eb52-7f88-474d-bdec-a81890176513 None admin] RBAC: Adding query filter params (role.id=dc7cfc688d354df4bc870fe8621c48bd, scope.project.id=c3a5c629197a4f8ba3a3d1ad27faaa7e, user.id=92cdd324cc6545ae9e3f84678c6b4254) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:07.605640 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f285eb52-7f88-474d-bdec-a81890176513 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:07.608332 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f285eb52-7f88-474d-bdec-a81890176513 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:07.616572 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 113/225] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:19:07 2022] GET /identity/v3/role_assignments?user.id=92cdd324cc6545ae9e3f84678c6b4254&scope.project.id=c3a5c629197a4f8ba3a3d1ad27faaa7e&role.id=dc7cfc688d354df4bc870fe8621c48bd => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:07.618157 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f285eb52-7f88-474d-bdec-a81890176513 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:07.650419 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:07.760376 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114e451-4729-4c1d-b7c7-3998d2c77a06 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:07.804848 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:07.823694 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f285eb52-7f88-474d-bdec-a81890176513 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:07.829690 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2584bc17-5a41-41a7-9001-f40198a02af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:07.830361 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584bc17-5a41-41a7-9001-f40198a02af8 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:07.830641 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584bc17-5a41-41a7-9001-f40198a02af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:07.830916 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584bc17-5a41-41a7-9001-f40198a02af8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:07.845103 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2584bc17-5a41-41a7-9001-f40198a02af8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:07.851257 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-2584bc17-5a41-41a7-9001-f40198a02af8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:19:07.851257 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:07.851257 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:07.851257 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:07.851257 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:07.851257 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:07.851257 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:07.851257 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:07.851257 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:07.851257 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:07.851257 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:07.851257 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:07.851257 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:07.851257 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:07.851257 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:07.851257 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:07.852127 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:07.852127 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:07.852127 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:07.852127 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:07.852127 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:07.852127 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:07.852127 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:07.852127 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:07.852127 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:07.852127 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:07.852127 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:07.852127 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:07.852127 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:07.852127 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:07.852127 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:07.852127 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:07.852942 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:07.852942 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:07.852942 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:07.852942 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:07.852942 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:07.852942 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:07.852942 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:07.852942 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:07.852942 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:07.852942 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:07.852942 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:19:07.852942 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:07.853555 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 113/226] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:19:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:07.858548 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2584bc17-5a41-41a7-9001-f40198a02af8 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:07.868019 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a2110-9c3f-4068-8540-56b238da0bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:07.868980 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a2110-9c3f-4068-8540-56b238da0bcf None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:07.869410 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a2110-9c3f-4068-8540-56b238da0bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:07.869818 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a2110-9c3f-4068-8540-56b238da0bcf None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:07.870820 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a2110-9c3f-4068-8540-56b238da0bcf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:07.871383 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a2110-9c3f-4068-8540-56b238da0bcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:07.873440 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a2110-9c3f-4068-8540-56b238da0bcf None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:07.882535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 114/227] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:19:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:07.887947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a2110-9c3f-4068-8540-56b238da0bcf None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:08.024405 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2584bc17-5a41-41a7-9001-f40198a02af8 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:08.045752 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:08.084847 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a2110-9c3f-4068-8540-56b238da0bcf None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:08.088846 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e625c9-4199-4248-9779-33a2ff33b734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:08.089692 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e625c9-4199-4248-9779-33a2ff33b734 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:08.090196 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e625c9-4199-4248-9779-33a2ff33b734 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:08.090683 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e625c9-4199-4248-9779-33a2ff33b734 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:08.092111 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e625c9-4199-4248-9779-33a2ff33b734 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:08.092758 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e625c9-4199-4248-9779-33a2ff33b734 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:08.095254 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e625c9-4199-4248-9779-33a2ff33b734 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:08.114501 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 114/228] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:19:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:08.117789 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e625c9-4199-4248-9779-33a2ff33b734 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:08.157034 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e860b723-d7a1-4fc4-b3bb-a1502f1a6ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:08.158215 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e860b723-d7a1-4fc4-b3bb-a1502f1a6ada None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:08.161367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e860b723-d7a1-4fc4-b3bb-a1502f1a6ada None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:08.161970 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e860b723-d7a1-4fc4-b3bb-a1502f1a6ada None admin] PATH_INFO: `/v3/users/b6f92cb9fa9a479e88be642cebb82831` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:08.163673 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e860b723-d7a1-4fc4-b3bb-a1502f1a6ada None admin] RBAC: Authorizing `identity:get_user(user_id=b6f92cb9fa9a479e88be642cebb82831)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:08.165724 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e860b723-d7a1-4fc4-b3bb-a1502f1a6ada None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:08.174514 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 115/229] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:19:07 2022] GET /identity/v3/users/b6f92cb9fa9a479e88be642cebb82831 => generated 283 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:08.179651 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e860b723-d7a1-4fc4-b3bb-a1502f1a6ada None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:08.226050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:08.269852 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e625c9-4199-4248-9779-33a2ff33b734 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:08.304548 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ec348-fad6-4811-9590-2e82c10e3fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:08.305200 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ec348-fad6-4811-9590-2e82c10e3fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:08.305478 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ec348-fad6-4811-9590-2e82c10e3fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:08.305745 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ec348-fad6-4811-9590-2e82c10e3fe3 None admin] PATH_INFO: `/v3/users/b6f92cb9fa9a479e88be642cebb82831` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:08.306994 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5ec348-fad6-4811-9590-2e82c10e3fe3 None admin] RBAC: Authorizing `identity:get_user(user_id=b6f92cb9fa9a479e88be642cebb82831)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:08.308481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5ec348-fad6-4811-9590-2e82c10e3fe3 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:08.310965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 115/230] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:19:08 2022] GET /identity/v3/users/b6f92cb9fa9a479e88be642cebb82831 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:08.316524 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ec348-fad6-4811-9590-2e82c10e3fe3 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:08.363761 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:08.380691 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e860b723-d7a1-4fc4-b3bb-a1502f1a6ada None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:08.416835 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be7abd-0f99-4a07-b7d3-d1d4ea5e7406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:08.417474 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be7abd-0f99-4a07-b7d3-d1d4ea5e7406 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:08.417756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be7abd-0f99-4a07-b7d3-d1d4ea5e7406 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:08.418047 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be7abd-0f99-4a07-b7d3-d1d4ea5e7406 None admin] PATH_INFO: `/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:08.419202 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be7abd-0f99-4a07-b7d3-d1d4ea5e7406 None admin] RBAC: Authorizing `identity:get_project(project_id=c3a5c629197a4f8ba3a3d1ad27faaa7e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:08.421364 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be7abd-0f99-4a07-b7d3-d1d4ea5e7406 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:08.423468 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 116/231] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:19:08 2022] GET /identity/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:08.435755 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be7abd-0f99-4a07-b7d3-d1d4ea5e7406 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:08.444259 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ec348-fad6-4811-9590-2e82c10e3fe3 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:08.477834 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8857e-e2e1-408e-9cfe-7f5a54a8ece4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:08.478828 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8857e-e2e1-408e-9cfe-7f5a54a8ece4 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:08.479109 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8857e-e2e1-408e-9cfe-7f5a54a8ece4 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:08.479400 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8857e-e2e1-408e-9cfe-7f5a54a8ece4 None admin] PATH_INFO: `/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:08.486709 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8857e-e2e1-408e-9cfe-7f5a54a8ece4 None admin] RBAC: Authorizing `identity:get_project(project_id=c3a5c629197a4f8ba3a3d1ad27faaa7e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:08.488162 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8857e-e2e1-408e-9cfe-7f5a54a8ece4 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:08.489805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 116/232] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:19:08 2022] GET /identity/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:08.495300 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8857e-e2e1-408e-9cfe-7f5a54a8ece4 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:08.598444 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8857e-e2e1-408e-9cfe-7f5a54a8ece4 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:08.608445 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:08.618480 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:08.633109 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be7abd-0f99-4a07-b7d3-d1d4ea5e7406 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:08.650723 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97abb5ce-507d-4953-b0ed-1bec14c5e794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:08.651623 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97abb5ce-507d-4953-b0ed-1bec14c5e794 None admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:08.651731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97abb5ce-507d-4953-b0ed-1bec14c5e794 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:08.651924 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97abb5ce-507d-4953-b0ed-1bec14c5e794 None admin] PATH_INFO: `/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e/users/b6f92cb9fa9a479e88be642cebb82831/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:08.658795 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97abb5ce-507d-4953-b0ed-1bec14c5e794 None admin] RBAC: Authorizing `identity:create_grant(project_id=c3a5c629197a4f8ba3a3d1ad27faaa7e, user_id=b6f92cb9fa9a479e88be642cebb82831, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:08.660243 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97abb5ce-507d-4953-b0ed-1bec14c5e794 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:08.670784 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c79ea-0aa6-4701-9200-49c8de465a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:08.671714 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c79ea-0aa6-4701-9200-49c8de465a4a None admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:08.672201 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c79ea-0aa6-4701-9200-49c8de465a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:08.672705 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c79ea-0aa6-4701-9200-49c8de465a4a None admin] PATH_INFO: `/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e/users/b6f92cb9fa9a479e88be642cebb82831/roles/d1109db57b004dd9a4f84a634ff76205` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:08.675480 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c79ea-0aa6-4701-9200-49c8de465a4a None admin] RBAC: Authorizing `identity:create_grant(project_id=c3a5c629197a4f8ba3a3d1ad27faaa7e, user_id=b6f92cb9fa9a479e88be642cebb82831, role_id=d1109db57b004dd9a4f84a634ff76205)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:08.677208 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c79ea-0aa6-4701-9200-49c8de465a4a None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:08.685012 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 117/233] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 10:19:08 2022] PUT /identity/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e/users/b6f92cb9fa9a479e88be642cebb82831/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:19:08.692321 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 117/234] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 10:19:08 2022] PUT /identity/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e/users/b6f92cb9fa9a479e88be642cebb82831/roles/d1109db57b004dd9a4f84a634ff76205 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:19:09.692324 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2bc43-c025-488a-bab2-997283ab8854 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:09.692877 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2bc43-c025-488a-bab2-997283ab8854 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:09.692877 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2bc43-c025-488a-bab2-997283ab8854 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:09.694233 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 118/235] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:09.710743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8654022-c237-4fe5-a382-4e0fbf4a879c None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:09.711230 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8654022-c237-4fe5-a382-4e0fbf4a879c None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:09.711330 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8654022-c237-4fe5-a382-4e0fbf4a879c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:09.739100 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-c8654022-c237-4fe5-a382-4e0fbf4a879c None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:09.740300 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-c8654022-c237-4fe5-a382-4e0fbf4a879c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:09.838241 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 118/236] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:09.880140 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f5a71-4069-426b-808a-3174551fb6a8 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:09.880443 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f5a71-4069-426b-808a-3174551fb6a8 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:09.880679 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f5a71-4069-426b-808a-3174551fb6a8 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:09.882321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 119/237] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:09.887615 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec69f1-1494-489b-9c90-20843f8ddbfe None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:09.887904 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec69f1-1494-489b-9c90-20843f8ddbfe None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:09.888697 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec69f1-1494-489b-9c90-20843f8ddbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:09.914363 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-18ec69f1-1494-489b-9c90-20843f8ddbfe None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:09.915570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-18ec69f1-1494-489b-9c90-20843f8ddbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:10.012452 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 119/238] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:10.145649 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f5a71-4069-426b-808a-3174551fb6a8 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:10.274850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f5a71-4069-426b-808a-3174551fb6a8 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:10.310060 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f6705-8d18-4ab0-9165-15a608eb301c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:10.310697 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f6705-8d18-4ab0-9165-15a608eb301c None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:10.311018 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f6705-8d18-4ab0-9165-15a608eb301c None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:10.311288 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f6705-8d18-4ab0-9165-15a608eb301c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:10.319858 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed503ab0-2827-4d12-95ed-91943e249538 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:10.320143 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed503ab0-2827-4d12-95ed-91943e249538 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:10.320426 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed503ab0-2827-4d12-95ed-91943e249538 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:10.321928 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 120/239] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:10.324865 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5f6705-8d18-4ab0-9165-15a608eb301c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:10.330329 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-5f5f6705-8d18-4ab0-9165-15a608eb301c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 10:19:10.330329 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:10.330329 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:10.330329 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:10.330329 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:10.330329 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:10.330329 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:10.330329 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:10.330329 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:10.330329 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:10.330329 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:10.330329 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:10.330329 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:10.330329 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:10.330329 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:10.330329 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:10.331204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:10.331204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:10.331204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:10.331204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:10.331204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:10.331204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:10.331204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:10.331204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:10.331204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:10.331204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:10.331204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:10.331204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:10.331204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:10.331204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:10.331204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:10.331204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:10.332010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:10.332010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:10.332010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:10.332010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:10.332010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:10.332010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:10.332010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:10.332010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:10.332010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:10.332010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:10.332010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 10:19:10.332010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:10.333883 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 120/240] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:19:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:10.336955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f7c9f9-ef0e-464b-b549-f9d409237550 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:10.336955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f7c9f9-ef0e-464b-b549-f9d409237550 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:10.336955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f7c9f9-ef0e-464b-b549-f9d409237550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:10.339907 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f6705-8d18-4ab0-9165-15a608eb301c None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:10.374728 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-15f7c9f9-ef0e-464b-b549-f9d409237550 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:10.377229 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-15f7c9f9-ef0e-464b-b549-f9d409237550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:10.382307 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:10.394873 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:10.494576 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 121/241] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 10:19:10.495768 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f7c9f9-ef0e-464b-b549-f9d409237550 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:10.504198 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f6705-8d18-4ab0-9165-15a608eb301c None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:10.540404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5abb74-5e5b-4656-884e-c4e19dfaa911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:10.540945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5abb74-5e5b-4656-884e-c4e19dfaa911 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:10.541225 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5abb74-5e5b-4656-884e-c4e19dfaa911 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:10.541499 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5abb74-5e5b-4656-884e-c4e19dfaa911 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:10.542194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5abb74-5e5b-4656-884e-c4e19dfaa911 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:10.542497 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5abb74-5e5b-4656-884e-c4e19dfaa911 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:10.543873 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5abb74-5e5b-4656-884e-c4e19dfaa911 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:10.568041 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 121/242] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 10:19:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:10.579048 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5abb74-5e5b-4656-884e-c4e19dfaa911 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:10.650049 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f7c9f9-ef0e-464b-b549-f9d409237550 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:10.705803 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d86a1-cb1a-4e9b-b133-9f9333847595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:10.706470 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d86a1-cb1a-4e9b-b133-9f9333847595 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:10.706748 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d86a1-cb1a-4e9b-b133-9f9333847595 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:10.707021 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d86a1-cb1a-4e9b-b133-9f9333847595 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:10.714045 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329d86a1-cb1a-4e9b-b133-9f9333847595 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:10.714045 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329d86a1-cb1a-4e9b-b133-9f9333847595 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:10.714636 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 122/243] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:10.723100 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d86a1-cb1a-4e9b-b133-9f9333847595 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:10.742394 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:10.887617 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:10.895513 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d86a1-cb1a-4e9b-b133-9f9333847595 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:10.917769 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5abb74-5e5b-4656-884e-c4e19dfaa911 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:10.932640 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2384325-4894-4c60-a007-ac304eaf8b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:10.933448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2384325-4894-4c60-a007-ac304eaf8b61 None admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:10.933862 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2384325-4894-4c60-a007-ac304eaf8b61 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:10.934303 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2384325-4894-4c60-a007-ac304eaf8b61 None admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:10.935112 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2384325-4894-4c60-a007-ac304eaf8b61 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:10.936694 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2384325-4894-4c60-a007-ac304eaf8b61 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:10.965867 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 123/244] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 10:19:10 2022] POST /identity/v3/users => generated 293 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:19:11.004899 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c8e86-838c-425b-b121-7700bf7ce495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:11.005879 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c8e86-838c-425b-b121-7700bf7ce495 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:11.006309 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c8e86-838c-425b-b121-7700bf7ce495 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:11.006741 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c8e86-838c-425b-b121-7700bf7ce495 None admin] PATH_INFO: `/v3/users/15dd612d16de458186fea2609c7fdf41` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:11.027558 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c8e86-838c-425b-b121-7700bf7ce495 None admin] RBAC: Authorizing `identity:get_user(user_id=15dd612d16de458186fea2609c7fdf41)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:11.029849 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c8e86-838c-425b-b121-7700bf7ce495 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:11.031881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 122/245] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:19:10 2022] GET /identity/v3/users/15dd612d16de458186fea2609c7fdf41 => generated 297 bytes in 453 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:11.050858 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2384325-4894-4c60-a007-ac304eaf8b61 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:11.110234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c8e86-838c-425b-b121-7700bf7ce495 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:11.190253 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:11.284126 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2384325-4894-4c60-a007-ac304eaf8b61 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:11.327544 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089bd6a2-dcd4-46f7-bee2-6f8047319c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:11.328513 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089bd6a2-dcd4-46f7-bee2-6f8047319c6d None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:11.329008 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089bd6a2-dcd4-46f7-bee2-6f8047319c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:11.329717 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089bd6a2-dcd4-46f7-bee2-6f8047319c6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:11.331373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c8e86-838c-425b-b121-7700bf7ce495 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:11.390680 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c70422-e124-419a-b8a6-c51e7a9c1916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:11.391241 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c70422-e124-419a-b8a6-c51e7a9c1916 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:11.391521 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c70422-e124-419a-b8a6-c51e7a9c1916 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:11.391795 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c70422-e124-419a-b8a6-c51e7a9c1916 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:11.409186 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c70422-e124-419a-b8a6-c51e7a9c1916 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:11.412672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-d0c70422-e124-419a-b8a6-c51e7a9c1916 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:19:11.412672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:11.412672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:11.412672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:11.412672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:11.412672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:11.412672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:11.412672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:11.412672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:11.412672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:11.412672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:11.412672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:11.412672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:11.412672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:11.412672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:11.412672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:11.413601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:11.413601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:11.413601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:11.413601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:11.413601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:11.413601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:11.413601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:11.413601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:11.413601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:11.413601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:11.413601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:11.413601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:11.413601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:11.413601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:11.413601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:11.413601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:11.414421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:11.414421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:11.414421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:11.414421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:11.414421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:11.414421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:11.414421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:11.414421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:11.414421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:11.414421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:11.414421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:19:11.414421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:11.414421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 123/246] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:19:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:11.415090 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28aac1-404f-4cda-8bdb-029a2fbd71b8 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:11.415955 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28aac1-404f-4cda-8bdb-029a2fbd71b8 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:11.415955 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28aac1-404f-4cda-8bdb-029a2fbd71b8 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:11.420545 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 124/247] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:11.425561 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28aac1-404f-4cda-8bdb-029a2fbd71b8 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:11.434631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089bd6a2-dcd4-46f7-bee2-6f8047319c6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:11.436607 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089bd6a2-dcd4-46f7-bee2-6f8047319c6d None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:11.437662 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:11.515360 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:11.562313 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28aac1-404f-4cda-8bdb-029a2fbd71b8 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:11.562923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 124/248] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:19:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 10:19:11.564296 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d957a0-28d1-4e2a-b70d-9fb6709a6487 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:11.564580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d957a0-28d1-4e2a-b70d-9fb6709a6487 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:11.564961 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d957a0-28d1-4e2a-b70d-9fb6709a6487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:11.583196 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-69d957a0-28d1-4e2a-b70d-9fb6709a6487 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:11.584395 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-69d957a0-28d1-4e2a-b70d-9fb6709a6487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:11.609926 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c02060-d37c-4a07-8afa-1cef27abdf72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:11.611136 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c02060-d37c-4a07-8afa-1cef27abdf72 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:11.611992 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c02060-d37c-4a07-8afa-1cef27abdf72 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:11.612738 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c02060-d37c-4a07-8afa-1cef27abdf72 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:11.614049 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c02060-d37c-4a07-8afa-1cef27abdf72 None admin] RBAC: Adding query filter params (name=member) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:11.614921 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c02060-d37c-4a07-8afa-1cef27abdf72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:11.617026 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c02060-d37c-4a07-8afa-1cef27abdf72 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:11.628797 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 125/249] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 10:19:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:11.630947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19d999-c6f4-40ad-9a71-9ae63df6e767 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:11.631427 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19d999-c6f4-40ad-9a71-9ae63df6e767 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:11.631924 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19d999-c6f4-40ad-9a71-9ae63df6e767 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:11.634556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 126/250] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:11.635687 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e19d999-c6f4-40ad-9a71-9ae63df6e767 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:11.659924 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 125/251] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:11.660596 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d957a0-28d1-4e2a-b70d-9fb6709a6487 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:11.727637 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:11.796397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e19d999-c6f4-40ad-9a71-9ae63df6e767 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:11.860132 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca56ea9-76e0-4e4f-9f0e-046b21415b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:11.860771 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca56ea9-76e0-4e4f-9f0e-046b21415b29 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:11.861052 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca56ea9-76e0-4e4f-9f0e-046b21415b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:11.862245 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca56ea9-76e0-4e4f-9f0e-046b21415b29 None admin] PATH_INFO: `/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:11.863679 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:11.864112 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca56ea9-76e0-4e4f-9f0e-046b21415b29 None admin] RBAC: Authorizing `identity:get_project(project_id=c3a5c629197a4f8ba3a3d1ad27faaa7e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:11.865217 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca56ea9-76e0-4e4f-9f0e-046b21415b29 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:11.875296 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 127/252] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:19:11 2022] GET /identity/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:11.876596 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec3457-ffdd-4d99-9f19-0be8b3f293c4 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:11.876880 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec3457-ffdd-4d99-9f19-0be8b3f293c4 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:11.877159 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec3457-ffdd-4d99-9f19-0be8b3f293c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:11.886616 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d957a0-28d1-4e2a-b70d-9fb6709a6487 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:11.911586 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-2aec3457-ffdd-4d99-9f19-0be8b3f293c4 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:11.912703 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-2aec3457-ffdd-4d99-9f19-0be8b3f293c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:11.951271 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366cbf4-4477-42d8-a60e-20922ddbe274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:11.952241 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366cbf4-4477-42d8-a60e-20922ddbe274 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:11.952747 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366cbf4-4477-42d8-a60e-20922ddbe274 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:11.953232 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366cbf4-4477-42d8-a60e-20922ddbe274 None admin] PATH_INFO: `/v3/users/92cdd324cc6545ae9e3f84678c6b4254` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:11.958680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6366cbf4-4477-42d8-a60e-20922ddbe274 None admin] RBAC: Authorizing `identity:get_user(user_id=92cdd324cc6545ae9e3f84678c6b4254)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:11.960352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6366cbf4-4477-42d8-a60e-20922ddbe274 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:11.962553 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 126/253] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:19:11 2022] GET /identity/v3/users/92cdd324cc6545ae9e3f84678c6b4254 => generated 297 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:11.963548 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366cbf4-4477-42d8-a60e-20922ddbe274 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:11.990137 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 128/254] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:11.990762 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec3457-ffdd-4d99-9f19-0be8b3f293c4 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:12.062387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:12.071601 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366cbf4-4477-42d8-a60e-20922ddbe274 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:12.103103 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec3457-ffdd-4d99-9f19-0be8b3f293c4 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:12.108028 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f07e4f-b6e8-4262-b738-75517f5abb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:12.108908 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f07e4f-b6e8-4262-b738-75517f5abb18 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:12.109377 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f07e4f-b6e8-4262-b738-75517f5abb18 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:12.109838 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f07e4f-b6e8-4262-b738-75517f5abb18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:12.110778 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f07e4f-b6e8-4262-b738-75517f5abb18 None admin] RBAC: Adding query filter params (role.id=4ffa6e25be8941078cca7449c84e50d5, scope.project.id=c3a5c629197a4f8ba3a3d1ad27faaa7e, user.id=15dd612d16de458186fea2609c7fdf41) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:12.111286 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f07e4f-b6e8-4262-b738-75517f5abb18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:12.112941 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f07e4f-b6e8-4262-b738-75517f5abb18 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:12.119193 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 127/255] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:19:11 2022] GET /identity/v3/role_assignments?user.id=15dd612d16de458186fea2609c7fdf41&scope.project.id=c3a5c629197a4f8ba3a3d1ad27faaa7e&role.id=4ffa6e25be8941078cca7449c84e50d5 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:12.120246 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f07e4f-b6e8-4262-b738-75517f5abb18 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:12.144405 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84fe1fc-69de-4790-ad3b-06ee9bb43e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:12.145046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84fe1fc-69de-4790-ad3b-06ee9bb43e3b None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:12.145325 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84fe1fc-69de-4790-ad3b-06ee9bb43e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:12.145594 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84fe1fc-69de-4790-ad3b-06ee9bb43e3b None admin] PATH_INFO: `/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:12.146806 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84fe1fc-69de-4790-ad3b-06ee9bb43e3b None admin] RBAC: Authorizing `identity:get_project(project_id=c3a5c629197a4f8ba3a3d1ad27faaa7e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:12.148271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84fe1fc-69de-4790-ad3b-06ee9bb43e3b None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:12.150014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 129/256] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:19:11 2022] GET /identity/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:12.156716 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84fe1fc-69de-4790-ad3b-06ee9bb43e3b None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:12.160371 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:12.262082 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f07e4f-b6e8-4262-b738-75517f5abb18 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:12.282319 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84fe1fc-69de-4790-ad3b-06ee9bb43e3b None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:12.307272 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0495cc8f-fdee-4823-bef4-25f3cd32b982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:12.308208 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495cc8f-fdee-4823-bef4-25f3cd32b982 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:12.308555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495cc8f-fdee-4823-bef4-25f3cd32b982 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:12.308895 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495cc8f-fdee-4823-bef4-25f3cd32b982 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:12.321329 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519e8a1-5b16-499e-9f05-20d16e37098d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:12.322349 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519e8a1-5b16-499e-9f05-20d16e37098d None admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:12.322798 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519e8a1-5b16-499e-9f05-20d16e37098d None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:12.323266 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519e8a1-5b16-499e-9f05-20d16e37098d None admin] PATH_INFO: `/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e/users/92cdd324cc6545ae9e3f84678c6b4254/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:12.323951 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0495cc8f-fdee-4823-bef4-25f3cd32b982 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:12.326251 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0519e8a1-5b16-499e-9f05-20d16e37098d None admin] RBAC: Authorizing `identity:create_grant(project_id=c3a5c629197a4f8ba3a3d1ad27faaa7e, user_id=92cdd324cc6545ae9e3f84678c6b4254, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:12.327954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0519e8a1-5b16-499e-9f05-20d16e37098d None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:12.330402 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-0495cc8f-fdee-4823-bef4-25f3cd32b982 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 10:19:12.330402 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:12.330402 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:12.330402 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:12.330402 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:12.330402 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:12.330402 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:12.330402 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:12.330402 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:12.330402 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:12.330402 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:12.330402 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:12.330402 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:12.330402 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:12.330402 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:12.330402 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:12.331261 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:12.331261 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:12.331261 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:12.331261 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:12.331261 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:12.331261 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:12.331261 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:12.331261 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:12.331261 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:12.331261 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:12.331261 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:12.331261 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:12.331261 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:12.331261 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:12.331261 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:12.331261 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:12.332062 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:12.332062 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:12.332062 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:12.332062 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:12.332062 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:12.332062 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:12.332062 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:12.332062 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:12.332062 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:12.332062 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:12.332062 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 10:19:12.332062 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:12.332062 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 128/257] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 10:19:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:12.336582 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:12.340321 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0495cc8f-fdee-4823-bef4-25f3cd32b982 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:12.351840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 130/258] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 10:19:12 2022] PUT /identity/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e/users/92cdd324cc6545ae9e3f84678c6b4254/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:19:12.462818 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519e8a1-5b16-499e-9f05-20d16e37098d None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:12.478399 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:12.519156 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0495cc8f-fdee-4823-bef4-25f3cd32b982 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:12.591064 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb922ed6-d0e3-4df4-b408-59e6fbfdd988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:12.591721 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb922ed6-d0e3-4df4-b408-59e6fbfdd988 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:12.592000 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb922ed6-d0e3-4df4-b408-59e6fbfdd988 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:12.592267 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb922ed6-d0e3-4df4-b408-59e6fbfdd988 None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:12.593065 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb922ed6-d0e3-4df4-b408-59e6fbfdd988 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:12.593239 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb922ed6-d0e3-4df4-b408-59e6fbfdd988 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:12.594675 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb922ed6-d0e3-4df4-b408-59e6fbfdd988 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:12.603924 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 129/259] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:19:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:12.619981 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb922ed6-d0e3-4df4-b408-59e6fbfdd988 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:12.645504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519e8a1-5b16-499e-9f05-20d16e37098d None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:12.682390 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b7cd9-416a-4b1d-8285-d78f7ac65497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:12.683390 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b7cd9-416a-4b1d-8285-d78f7ac65497 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:12.683901 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b7cd9-416a-4b1d-8285-d78f7ac65497 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:12.684416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b7cd9-416a-4b1d-8285-d78f7ac65497 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:12.696073 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b7cd9-416a-4b1d-8285-d78f7ac65497 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:12.703276 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-e68b7cd9-416a-4b1d-8285-d78f7ac65497 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:19:12.703276 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:12.703276 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:12.703276 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:12.703276 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:12.703276 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:12.703276 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:12.703276 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:12.703276 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:12.703276 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:12.703276 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:12.703276 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:12.703276 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:12.703276 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:12.703276 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:12.703276 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:12.704252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:12.704252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:12.704252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:12.704252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:12.704252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:12.704252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:12.704252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:12.704252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:12.704252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:12.704252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:12.704252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:12.704252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:12.704252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:12.704252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:12.704252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:12.704252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:12.705320 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:12.705320 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:12.705320 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:12.705320 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:12.705320 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:12.705320 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:12.705320 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:12.705320 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:12.705320 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:12.705320 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:12.705320 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:19:12.705320 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:12.707902 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 131/260] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:19:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:19:12.726334 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b7cd9-416a-4b1d-8285-d78f7ac65497 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:12.749653 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb922ed6-d0e3-4df4-b408-59e6fbfdd988 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:12.785156 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ecbef4-e1d8-4898-bdcf-d470225aa821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:12.785719 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecbef4-e1d8-4898-bdcf-d470225aa821 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:12.785996 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecbef4-e1d8-4898-bdcf-d470225aa821 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:12.786290 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecbef4-e1d8-4898-bdcf-d470225aa821 None admin] PATH_INFO: `/v3/users/b6f92cb9fa9a479e88be642cebb82831` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:12.787421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ecbef4-e1d8-4898-bdcf-d470225aa821 None admin] RBAC: Authorizing `identity:get_user(user_id=b6f92cb9fa9a479e88be642cebb82831)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:12.792986 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ecbef4-e1d8-4898-bdcf-d470225aa821 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:12.792986 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 130/261] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:19:12 2022] GET /identity/v3/users/b6f92cb9fa9a479e88be642cebb82831 => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:12.800536 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ecbef4-e1d8-4898-bdcf-d470225aa821 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:12.830093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:12.830869 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:12.928207 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b7cd9-416a-4b1d-8285-d78f7ac65497 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:12.995468 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec62f35-e91f-4d62-891a-3758f2313026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:12.996083 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec62f35-e91f-4d62-891a-3758f2313026 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:12.996356 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec62f35-e91f-4d62-891a-3758f2313026 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:12.996630 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec62f35-e91f-4d62-891a-3758f2313026 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:12.997293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec62f35-e91f-4d62-891a-3758f2313026 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:12.998202 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec62f35-e91f-4d62-891a-3758f2313026 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:12.999575 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec62f35-e91f-4d62-891a-3758f2313026 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:13.010596 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ecbef4-e1d8-4898-bdcf-d470225aa821 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:13.024271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 132/262] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:19:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:13.033273 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec62f35-e91f-4d62-891a-3758f2313026 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:13.091396 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f19e1-194a-43b8-bffd-8f3eff5e427f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:13.092337 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f19e1-194a-43b8-bffd-8f3eff5e427f None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:13.092839 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f19e1-194a-43b8-bffd-8f3eff5e427f None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:13.093322 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f19e1-194a-43b8-bffd-8f3eff5e427f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:13.260876 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec62f35-e91f-4d62-891a-3758f2313026 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:13.261927 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31f19e1-194a-43b8-bffd-8f3eff5e427f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:13.263474 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31f19e1-194a-43b8-bffd-8f3eff5e427f None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:13.295655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:13.301467 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:13.327145 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da7ec6-ad47-4858-b665-84e6e47ccb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:13.327789 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da7ec6-ad47-4858-b665-84e6e47ccb04 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:13.328227 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da7ec6-ad47-4858-b665-84e6e47ccb04 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:13.328501 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da7ec6-ad47-4858-b665-84e6e47ccb04 None admin] PATH_INFO: `/v3/users/b6f92cb9fa9a479e88be642cebb82831` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:13.338928 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7da7ec6-ad47-4858-b665-84e6e47ccb04 None admin] RBAC: Authorizing `identity:get_user(user_id=b6f92cb9fa9a479e88be642cebb82831)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:13.340778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7da7ec6-ad47-4858-b665-84e6e47ccb04 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:13.343382 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 133/263] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:19:13 2022] GET /identity/v3/users/b6f92cb9fa9a479e88be642cebb82831 => generated 283 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:13.349235 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da7ec6-ad47-4858-b665-84e6e47ccb04 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:13.505798 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 131/264] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:19:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 705 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 10:19:13.511205 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f19e1-194a-43b8-bffd-8f3eff5e427f None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:13.535899 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da7ec6-ad47-4858-b665-84e6e47ccb04 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:13.616327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede701d6-9e3a-4803-8218-a6eba21e018f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:13.616875 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede701d6-9e3a-4803-8218-a6eba21e018f None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:13.617156 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede701d6-9e3a-4803-8218-a6eba21e018f None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:13.617490 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede701d6-9e3a-4803-8218-a6eba21e018f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:13.724841 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f19e1-194a-43b8-bffd-8f3eff5e427f None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:13.741540 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:13.751635 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:13.771997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede701d6-9e3a-4803-8218-a6eba21e018f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:13.773889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede701d6-9e3a-4803-8218-a6eba21e018f None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:13.776953 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403eaf47-8ece-4352-b1a9-a6a394636e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:13.777588 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403eaf47-8ece-4352-b1a9-a6a394636e38 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:13.777862 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403eaf47-8ece-4352-b1a9-a6a394636e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:13.778155 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403eaf47-8ece-4352-b1a9-a6a394636e38 None admin] PATH_INFO: `/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:13.786912 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403eaf47-8ece-4352-b1a9-a6a394636e38 None admin] RBAC: Authorizing `identity:get_project(project_id=c3a5c629197a4f8ba3a3d1ad27faaa7e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:13.788384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403eaf47-8ece-4352-b1a9-a6a394636e38 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:13.790125 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 132/265] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:19:13 2022] GET /identity/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:13.801566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403eaf47-8ece-4352-b1a9-a6a394636e38 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:13.935298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 134/266] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:19:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 10:19:13.941900 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede701d6-9e3a-4803-8218-a6eba21e018f None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:14.012138 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403eaf47-8ece-4352-b1a9-a6a394636e38 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:14.035891 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:14.055138 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede701d6-9e3a-4803-8218-a6eba21e018f None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:14.090797 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364d980-6392-42ec-ab37-5882384a230f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:14.092122 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364d980-6392-42ec-ab37-5882384a230f None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:14.092498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364d980-6392-42ec-ab37-5882384a230f None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:14.092776 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364d980-6392-42ec-ab37-5882384a230f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:14.093504 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a364d980-6392-42ec-ab37-5882384a230f None admin] RBAC: Adding query filter params (role.id=d1109db57b004dd9a4f84a634ff76205, scope.project.id=c3a5c629197a4f8ba3a3d1ad27faaa7e, user.id=b6f92cb9fa9a479e88be642cebb82831) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:14.093808 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a364d980-6392-42ec-ab37-5882384a230f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:14.095479 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a364d980-6392-42ec-ab37-5882384a230f None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:14.103207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 133/267] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:19:13 2022] GET /identity/v3/role_assignments?user.id=b6f92cb9fa9a479e88be642cebb82831&scope.project.id=c3a5c629197a4f8ba3a3d1ad27faaa7e&role.id=d1109db57b004dd9a4f84a634ff76205 => generated 608 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:14.127155 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb9168-e260-4c52-8e3d-fa0e55820c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:14.128117 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb9168-e260-4c52-8e3d-fa0e55820c45 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:14.128539 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb9168-e260-4c52-8e3d-fa0e55820c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:14.128950 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb9168-e260-4c52-8e3d-fa0e55820c45 None admin] PATH_INFO: `/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:14.130464 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bb9168-e260-4c52-8e3d-fa0e55820c45 None admin] RBAC: Authorizing `identity:get_project(project_id=c3a5c629197a4f8ba3a3d1ad27faaa7e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:14.132148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bb9168-e260-4c52-8e3d-fa0e55820c45 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:14.134429 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 135/268] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:19:13 2022] GET /identity/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:14.139998 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364d980-6392-42ec-ab37-5882384a230f None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:14.165845 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d34d58-85ca-45db-8495-c5e501064606 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:14.166379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d34d58-85ca-45db-8495-c5e501064606 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:14.166853 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d34d58-85ca-45db-8495-c5e501064606 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:14.168343 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 136/269] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:19:14.179846 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81d562d-323f-4a4e-9efe-e37ac1f9fd7c None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:14.180152 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81d562d-323f-4a4e-9efe-e37ac1f9fd7c None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:14.180397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81d562d-323f-4a4e-9efe-e37ac1f9fd7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:14.217143 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:14.219284 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-f81d562d-323f-4a4e-9efe-e37ac1f9fd7c None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:14.222335 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-f81d562d-323f-4a4e-9efe-e37ac1f9fd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:14.322142 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 137/270] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 10:19:14.362854 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364d980-6392-42ec-ab37-5882384a230f None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:14.377425 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e360e1-3c3b-4fd1-b190-83b6c76da84f None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:14.377711 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e360e1-3c3b-4fd1-b190-83b6c76da84f None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:14.377982 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e360e1-3c3b-4fd1-b190-83b6c76da84f None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:14.379518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 138/271] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:19:14.385333 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1ea25-39f5-485c-8ef6-5cc3c24fea71 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:14.385635 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1ea25-39f5-485c-8ef6-5cc3c24fea71 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:14.385877 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1ea25-39f5-485c-8ef6-5cc3c24fea71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:14.408659 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-68c1ea25-39f5-485c-8ef6-5cc3c24fea71 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:14.410710 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-68c1ea25-39f5-485c-8ef6-5cc3c24fea71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:14.451459 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f135dd7-eed7-45b6-b491-f1fbc11d5202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:14.452067 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f135dd7-eed7-45b6-b491-f1fbc11d5202 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:14.452341 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f135dd7-eed7-45b6-b491-f1fbc11d5202 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:14.452608 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f135dd7-eed7-45b6-b491-f1fbc11d5202 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:14.453340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f135dd7-eed7-45b6-b491-f1fbc11d5202 None admin] RBAC: Adding query filter params (role.id=532b37d9076c45618cf352aafb85f35e, scope.project.id=c3a5c629197a4f8ba3a3d1ad27faaa7e, user.id=b6f92cb9fa9a479e88be642cebb82831) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:14.453644 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f135dd7-eed7-45b6-b491-f1fbc11d5202 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:14.455373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f135dd7-eed7-45b6-b491-f1fbc11d5202 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:14.473560 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 134/272] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:19:14 2022] GET /identity/v3/role_assignments?user.id=b6f92cb9fa9a479e88be642cebb82831&scope.project.id=c3a5c629197a4f8ba3a3d1ad27faaa7e&role.id=532b37d9076c45618cf352aafb85f35e => generated 608 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:14.517527 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 139/273] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 10:19:14.633193 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f135dd7-eed7-45b6-b491-f1fbc11d5202 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:14.744457 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f135dd7-eed7-45b6-b491-f1fbc11d5202 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:14.756569 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:14.785804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807bdfa3-e851-4ad4-9021-a1b94d934814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:14.786475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807bdfa3-e851-4ad4-9021-a1b94d934814 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:14.786755 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807bdfa3-e851-4ad4-9021-a1b94d934814 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:14.787104 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807bdfa3-e851-4ad4-9021-a1b94d934814 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:14.788318 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807bdfa3-e851-4ad4-9021-a1b94d934814 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:14.789804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807bdfa3-e851-4ad4-9021-a1b94d934814 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:14.791646 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 135/274] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:14.797400 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1ea25-39f5-485c-8ef6-5cc3c24fea71 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:14.817087 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:14.908435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1ea25-39f5-485c-8ef6-5cc3c24fea71 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:14.944118 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f167c-063b-4e5b-8d29-148466502365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:14.944757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f167c-063b-4e5b-8d29-148466502365 None admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:14.944997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f167c-063b-4e5b-8d29-148466502365 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:14.945264 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f167c-063b-4e5b-8d29-148466502365 None admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:14.945928 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097f167c-063b-4e5b-8d29-148466502365 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:14.947547 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097f167c-063b-4e5b-8d29-148466502365 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:14.977782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 140/275] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 10:19:14 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:19:15.002899 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807bdfa3-e851-4ad4-9021-a1b94d934814 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:15.120733 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807bdfa3-e851-4ad4-9021-a1b94d934814 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:15.160575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ec7d4-0f21-4683-80ea-c69bac49762b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:15.161612 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ec7d4-0f21-4683-80ea-c69bac49762b None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:15.162103 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ec7d4-0f21-4683-80ea-c69bac49762b None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:15.162569 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ec7d4-0f21-4683-80ea-c69bac49762b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:15.173597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ec7d4-0f21-4683-80ea-c69bac49762b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:15.179485 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-7e3ec7d4-0f21-4683-80ea-c69bac49762b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:19:15.179485 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:15.179485 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:15.179485 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:15.179485 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:15.179485 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:15.179485 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:15.179485 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:15.179485 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:15.179485 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:15.179485 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:15.179485 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:15.179485 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:15.179485 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:15.179485 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:15.179485 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:15.180460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:15.180460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:15.180460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:15.180460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:15.180460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:15.180460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:15.180460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:15.180460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:15.180460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:15.180460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:15.180460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:15.180460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:15.180460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:15.180460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:15.180460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:15.180460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:15.181542 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:15.181542 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:15.181542 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:15.181542 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:15.181542 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:15.181542 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:15.181542 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:15.181542 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:15.181542 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:15.181542 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:15.181542 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:19:15.181542 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:15.181542 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 136/276] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:19:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:15.186866 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f167c-063b-4e5b-8d29-148466502365 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:15.263838 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:15.305950 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f167c-063b-4e5b-8d29-148466502365 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:15.347728 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833297d4-3eb6-49bf-80a5-94bc6c7b12f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:15.348620 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833297d4-3eb6-49bf-80a5-94bc6c7b12f2 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:15.348896 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833297d4-3eb6-49bf-80a5-94bc6c7b12f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:15.349166 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833297d4-3eb6-49bf-80a5-94bc6c7b12f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:15.349828 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833297d4-3eb6-49bf-80a5-94bc6c7b12f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:15.350212 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833297d4-3eb6-49bf-80a5-94bc6c7b12f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:15.351605 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833297d4-3eb6-49bf-80a5-94bc6c7b12f2 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:15.362066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 141/277] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 10:19:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:15.367720 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ec7d4-0f21-4683-80ea-c69bac49762b None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:15.407873 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e583fc6-209a-4e18-8146-53741f37c350 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:15.408675 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e583fc6-209a-4e18-8146-53741f37c350 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:15.408675 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e583fc6-209a-4e18-8146-53741f37c350 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:15.410071 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 142/278] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:19:15.411130 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3ce73-a531-4644-9221-a36ec2f8655f None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:15.411409 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3ce73-a531-4644-9221-a36ec2f8655f None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:15.411685 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3ce73-a531-4644-9221-a36ec2f8655f None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:15.412886 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 143/279] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:15.413731 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:15.419307 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4cc44e-b96f-4891-8dec-2ea70a23fbab None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:15.420131 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4cc44e-b96f-4891-8dec-2ea70a23fbab None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:15.420404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4cc44e-b96f-4891-8dec-2ea70a23fbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:15.443446 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-7f4cc44e-b96f-4891-8dec-2ea70a23fbab None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:15.444659 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-7f4cc44e-b96f-4891-8dec-2ea70a23fbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:15.486054 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ec7d4-0f21-4683-80ea-c69bac49762b None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:15.531944 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b0e93-2c2e-4659-a067-a9d210af394b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:15.532494 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b0e93-2c2e-4659-a067-a9d210af394b None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:15.532770 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b0e93-2c2e-4659-a067-a9d210af394b None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:15.533041 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b0e93-2c2e-4659-a067-a9d210af394b None admin] PATH_INFO: `/v3/users/6eebdc2cfd194fdf8f8f72087e212dd1` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:15.537163 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 144/280] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:15.538444 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831688e-35f9-4093-a50a-5e87ab3ce80b None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:15.538720 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831688e-35f9-4093-a50a-5e87ab3ce80b None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:15.538987 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831688e-35f9-4093-a50a-5e87ab3ce80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:15.552654 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b0e93-2c2e-4659-a067-a9d210af394b None admin] RBAC: Authorizing `identity:get_user(user_id=6eebdc2cfd194fdf8f8f72087e212dd1)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:15.554516 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b0e93-2c2e-4659-a067-a9d210af394b None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:15.556738 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 137/281] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:19:15 2022] GET /identity/v3/users/6eebdc2cfd194fdf8f8f72087e212dd1 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:15.566145 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b0e93-2c2e-4659-a067-a9d210af394b None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:15.574710 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-9831688e-35f9-4093-a50a-5e87ab3ce80b None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:15.575943 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-9831688e-35f9-4093-a50a-5e87ab3ce80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:15.681629 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:15.709070 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 145/282] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:15.710694 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:15.725981 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b0e93-2c2e-4659-a067-a9d210af394b None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:15.784357 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d531c581-4a18-4b1a-8aa7-4f041c1e8c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:15.784876 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531c581-4a18-4b1a-8aa7-4f041c1e8c05 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:15.785151 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531c581-4a18-4b1a-8aa7-4f041c1e8c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:15.785416 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531c581-4a18-4b1a-8aa7-4f041c1e8c05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:15.786166 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d531c581-4a18-4b1a-8aa7-4f041c1e8c05 None admin] RBAC: Adding query filter params (role.id=dc7cfc688d354df4bc870fe8621c48bd, user.id=6eebdc2cfd194fdf8f8f72087e212dd1, scope.system=all) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:15.786467 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d531c581-4a18-4b1a-8aa7-4f041c1e8c05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:15.787894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d531c581-4a18-4b1a-8aa7-4f041c1e8c05 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:15.799211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 138/283] 10.222.0.49 () {56 vars in 1308 bytes} [Mon Sep 26 10:19:15 2022] GET /identity/v3/role_assignments?user.id=6eebdc2cfd194fdf8f8f72087e212dd1&scope.system=all&role.id=dc7cfc688d354df4bc870fe8621c48bd => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:15.883119 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831688e-35f9-4093-a50a-5e87ab3ce80b None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:16.034227 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831688e-35f9-4093-a50a-5e87ab3ce80b None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:16.074961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0419535-7b82-4f64-b0b5-d9c391d050a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:16.076478 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0419535-7b82-4f64-b0b5-d9c391d050a0 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:16.076948 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0419535-7b82-4f64-b0b5-d9c391d050a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:16.077424 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0419535-7b82-4f64-b0b5-d9c391d050a0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:16.088915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0419535-7b82-4f64-b0b5-d9c391d050a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:16.094915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-d0419535-7b82-4f64-b0b5-d9c391d050a0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 10:19:16.094915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:16.094915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:16.094915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:16.094915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:16.094915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:16.094915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:16.094915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:16.094915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:16.094915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:16.094915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:16.094915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:16.094915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:16.094915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:16.094915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:16.094915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:16.095754 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:16.095754 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:16.095754 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:16.095754 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:16.095754 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:16.095754 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:16.095754 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:16.095754 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:16.095754 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:16.095754 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:16.095754 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:16.095754 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:16.095754 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:16.095754 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:16.095754 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:16.095754 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:16.096535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:16.096535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:16.096535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:16.096535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:16.096535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:16.096535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:16.096535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:16.096535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:16.096535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:16.096535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:16.096535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 10:19:16.096535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:16.099140 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 146/284] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:19:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:16.107174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d531c581-4a18-4b1a-8aa7-4f041c1e8c05 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:16.151324 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0419535-7b82-4f64-b0b5-d9c391d050a0 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:16.260969 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:16.300137 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d531c581-4a18-4b1a-8aa7-4f041c1e8c05 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:16.344163 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0419535-7b82-4f64-b0b5-d9c391d050a0 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:16.380276 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702717d-85cd-4d69-804c-e98d7eba5da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:16.381239 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702717d-85cd-4d69-804c-e98d7eba5da4 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:16.381779 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702717d-85cd-4d69-804c-e98d7eba5da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:16.382278 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702717d-85cd-4d69-804c-e98d7eba5da4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:16.390762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a22ca-f6bd-4733-8e9c-9a2ea8785535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:16.393094 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a22ca-f6bd-4733-8e9c-9a2ea8785535 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:16.394650 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a22ca-f6bd-4733-8e9c-9a2ea8785535 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:16.396236 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a22ca-f6bd-4733-8e9c-9a2ea8785535 None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:16.396236 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a22ca-f6bd-4733-8e9c-9a2ea8785535 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:16.398286 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a22ca-f6bd-4733-8e9c-9a2ea8785535 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:16.399938 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a22ca-f6bd-4733-8e9c-9a2ea8785535 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:16.401321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702717d-85cd-4d69-804c-e98d7eba5da4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:16.408651 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-c702717d-85cd-4d69-804c-e98d7eba5da4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:19:16.408651 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:16.408651 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:16.408651 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:16.408651 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:16.408651 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:16.408651 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:16.408651 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:16.408651 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:16.408651 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:16.408651 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:16.408651 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:16.408651 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:16.408651 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:16.408651 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:16.408651 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:16.409496 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:16.409496 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:16.409496 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:16.409496 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:16.409496 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:16.409496 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:16.409496 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:16.409496 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:16.409496 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:16.409496 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:16.409496 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:16.409496 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:16.409496 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:16.409496 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:16.409496 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:16.409496 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:16.410321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:16.410321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:16.410321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:16.410321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:16.410321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:16.410321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:16.410321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:16.410321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:16.410321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:16.410321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:16.410321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:19:16.410321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:16.417673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 147/285] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:19:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:19:16.426043 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 139/286] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 10:19:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:16.427210 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702717d-85cd-4d69-804c-e98d7eba5da4 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:16.431515 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a22ca-f6bd-4733-8e9c-9a2ea8785535 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:16.478374 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:16.600034 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a22ca-f6bd-4733-8e9c-9a2ea8785535 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:16.643494 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702717d-85cd-4d69-804c-e98d7eba5da4 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:16.666756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6c60b-f192-40b2-a0eb-bd63891c79e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:16.667378 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6c60b-f192-40b2-a0eb-bd63891c79e1 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:16.667655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6c60b-f192-40b2-a0eb-bd63891c79e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:16.667922 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6c60b-f192-40b2-a0eb-bd63891c79e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:16.668786 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb6c60b-f192-40b2-a0eb-bd63891c79e1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:16.669104 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb6c60b-f192-40b2-a0eb-bd63891c79e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:16.670901 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb6c60b-f192-40b2-a0eb-bd63891c79e1 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:16.673591 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:16.682096 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 140/287] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 10:19:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:16.688370 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6c60b-f192-40b2-a0eb-bd63891c79e1 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:16.723788 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3899d714-1c41-4f86-8b74-737f18279763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:16.724468 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899d714-1c41-4f86-8b74-737f18279763 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:16.724751 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899d714-1c41-4f86-8b74-737f18279763 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:16.725018 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899d714-1c41-4f86-8b74-737f18279763 None admin] PATH_INFO: `/v3/users/15dd612d16de458186fea2609c7fdf41` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:16.726333 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3899d714-1c41-4f86-8b74-737f18279763 None admin] RBAC: Authorizing `identity:get_user(user_id=15dd612d16de458186fea2609c7fdf41)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:16.727761 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3899d714-1c41-4f86-8b74-737f18279763 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:16.729331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 148/288] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:19:16 2022] GET /identity/v3/users/15dd612d16de458186fea2609c7fdf41 => generated 297 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:16.735733 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3899d714-1c41-4f86-8b74-737f18279763 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:16.860683 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:16.865176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3899d714-1c41-4f86-8b74-737f18279763 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:16.896348 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6c60b-f192-40b2-a0eb-bd63891c79e1 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:16.909861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc66e1-1cf2-41be-9436-1f59bc17afc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:16.910805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc66e1-1cf2-41be-9436-1f59bc17afc0 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:16.911240 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc66e1-1cf2-41be-9436-1f59bc17afc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:16.911651 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc66e1-1cf2-41be-9436-1f59bc17afc0 None admin] PATH_INFO: `/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:16.912961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc66e1-1cf2-41be-9436-1f59bc17afc0 None admin] RBAC: Authorizing `identity:get_project(project_id=c3a5c629197a4f8ba3a3d1ad27faaa7e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:16.914593 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc66e1-1cf2-41be-9436-1f59bc17afc0 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:16.916486 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 149/289] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:19:16 2022] GET /identity/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:16.925272 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc66e1-1cf2-41be-9436-1f59bc17afc0 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:16.946161 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed438a-0814-40e5-91ef-580bcb010749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:16.946948 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed438a-0814-40e5-91ef-580bcb010749 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:16.947325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed438a-0814-40e5-91ef-580bcb010749 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:16.947696 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed438a-0814-40e5-91ef-580bcb010749 None admin] PATH_INFO: `/v3/users/92cdd324cc6545ae9e3f84678c6b4254` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:16.949143 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eed438a-0814-40e5-91ef-580bcb010749 None admin] RBAC: Authorizing `identity:get_user(user_id=92cdd324cc6545ae9e3f84678c6b4254)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:16.951326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eed438a-0814-40e5-91ef-580bcb010749 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:16.957935 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 141/290] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:19:16 2022] GET /identity/v3/users/92cdd324cc6545ae9e3f84678c6b4254 => generated 297 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:16.959556 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed438a-0814-40e5-91ef-580bcb010749 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:17.025767 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:17.094719 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed438a-0814-40e5-91ef-580bcb010749 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:17.131042 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb230371-58e7-4ea2-8401-ce806f8ae7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:17.132185 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb230371-58e7-4ea2-8401-ce806f8ae7ed None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:17.132588 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb230371-58e7-4ea2-8401-ce806f8ae7ed None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:17.132998 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb230371-58e7-4ea2-8401-ce806f8ae7ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:17.158374 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:17.168194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc66e1-1cf2-41be-9436-1f59bc17afc0 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:17.209862 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb230371-58e7-4ea2-8401-ce806f8ae7ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:17.211825 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb230371-58e7-4ea2-8401-ce806f8ae7ed None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:17.272924 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738faa63-2267-488f-a0cf-02015e947a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:17.273646 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738faa63-2267-488f-a0cf-02015e947a2f None admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:17.273928 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738faa63-2267-488f-a0cf-02015e947a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:17.274224 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738faa63-2267-488f-a0cf-02015e947a2f None admin] PATH_INFO: `/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e/users/15dd612d16de458186fea2609c7fdf41/roles/4ffa6e25be8941078cca7449c84e50d5` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:17.280001 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738faa63-2267-488f-a0cf-02015e947a2f None admin] RBAC: Authorizing `identity:create_grant(project_id=c3a5c629197a4f8ba3a3d1ad27faaa7e, user_id=15dd612d16de458186fea2609c7fdf41, role_id=4ffa6e25be8941078cca7449c84e50d5)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:17.281532 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738faa63-2267-488f-a0cf-02015e947a2f None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:17.310702 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 150/291] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 10:19:16 2022] PUT /identity/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e/users/15dd612d16de458186fea2609c7fdf41/roles/4ffa6e25be8941078cca7449c84e50d5 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:19:17.354449 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 142/292] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:19:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 10:19:17.360234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738faa63-2267-488f-a0cf-02015e947a2f None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:17.500269 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738faa63-2267-488f-a0cf-02015e947a2f None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:17.512122 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:17.542840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a6484-c28c-4562-891f-851f401bbd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:17.543528 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a6484-c28c-4562-891f-851f401bbd50 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:17.543809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a6484-c28c-4562-891f-851f401bbd50 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:17.544084 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a6484-c28c-4562-891f-851f401bbd50 None admin] PATH_INFO: `/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:17.546732 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8a6484-c28c-4562-891f-851f401bbd50 None admin] RBAC: Authorizing `identity:get_project(project_id=c3a5c629197a4f8ba3a3d1ad27faaa7e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:17.548170 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8a6484-c28c-4562-891f-851f401bbd50 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:17.549874 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 151/293] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:19:17 2022] GET /identity/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:17.556081 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb230371-58e7-4ea2-8401-ce806f8ae7ed None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:17.632728 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:17.689837 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb230371-58e7-4ea2-8401-ce806f8ae7ed None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:17.713705 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bc837-069a-415d-a3af-d64775f1ca61 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:17.714004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bc837-069a-415d-a3af-d64775f1ca61 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:17.714301 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bc837-069a-415d-a3af-d64775f1ca61 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:17.715936 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 152/294] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:19:17.722288 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de7c8c-438c-4e1d-9c92-c678f7e6a81d None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:17.722586 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de7c8c-438c-4e1d-9c92-c678f7e6a81d None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:17.722827 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de7c8c-438c-4e1d-9c92-c678f7e6a81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:17.727759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abbdcf4-24c9-4ac5-ae22-5df182d69c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:17.728376 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbdcf4-24c9-4ac5-ae22-5df182d69c6b None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:17.728654 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbdcf4-24c9-4ac5-ae22-5df182d69c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:17.728928 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbdcf4-24c9-4ac5-ae22-5df182d69c6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:17.729669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abbdcf4-24c9-4ac5-ae22-5df182d69c6b None admin] RBAC: Adding query filter params (role.id=dc7cfc688d354df4bc870fe8621c48bd, scope.project.id=c3a5c629197a4f8ba3a3d1ad27faaa7e, user.id=92cdd324cc6545ae9e3f84678c6b4254) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:17.729976 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abbdcf4-24c9-4ac5-ae22-5df182d69c6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:17.731562 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abbdcf4-24c9-4ac5-ae22-5df182d69c6b None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:17.739158 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 143/295] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:19:17 2022] GET /identity/v3/role_assignments?user.id=92cdd324cc6545ae9e3f84678c6b4254&scope.project.id=c3a5c629197a4f8ba3a3d1ad27faaa7e&role.id=dc7cfc688d354df4bc870fe8621c48bd => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:17.751009 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-17de7c8c-438c-4e1d-9c92-c678f7e6a81d None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:17.752148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-17de7c8c-438c-4e1d-9c92-c678f7e6a81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:17.777758 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a48424-c91f-4b77-a190-f58f891f0a7c None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:17.778106 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a48424-c91f-4b77-a190-f58f891f0a7c None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:17.779215 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a48424-c91f-4b77-a190-f58f891f0a7c None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:17.783465 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 144/296] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:19:17.788566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f16d5-6b25-4297-913b-e9bb1152952b None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:17.789039 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f16d5-6b25-4297-913b-e9bb1152952b None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:17.789458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f16d5-6b25-4297-913b-e9bb1152952b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:17.814953 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-427f16d5-6b25-4297-913b-e9bb1152952b None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:17.816882 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-427f16d5-6b25-4297-913b-e9bb1152952b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:17.863698 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 153/297] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 10:19:17.911753 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 145/298] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:18.144702 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17de7c8c-438c-4e1d-9c92-c678f7e6a81d None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:18.196013 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f16d5-6b25-4297-913b-e9bb1152952b None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:18.267248 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17de7c8c-438c-4e1d-9c92-c678f7e6a81d None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:18.276934 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:18.281539 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:18.321842 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40265620-23e2-47bf-ae5c-773c68e02519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:18.322858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40265620-23e2-47bf-ae5c-773c68e02519 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:18.323361 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40265620-23e2-47bf-ae5c-773c68e02519 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:18.323841 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40265620-23e2-47bf-ae5c-773c68e02519 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:18.336864 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40265620-23e2-47bf-ae5c-773c68e02519 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:18.342567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-40265620-23e2-47bf-ae5c-773c68e02519 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 10:19:18.342567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:18.342567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:18.342567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:18.342567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:18.342567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:18.342567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:18.342567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:18.342567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:18.342567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:18.342567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:18.342567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:18.342567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:18.342567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:18.342567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:18.342567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:18.343595 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:18.343595 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:18.343595 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:18.343595 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:18.343595 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:18.343595 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:18.343595 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:18.343595 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:18.343595 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:18.343595 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:18.343595 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:18.343595 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:18.343595 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:18.343595 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:18.343595 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:18.343595 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:18.344569 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:18.344569 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:18.344569 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:18.344569 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:18.344569 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:18.344569 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:18.344569 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:18.344569 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:18.344569 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:18.344569 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:18.344569 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 10:19:18.344569 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:18.345326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f16d5-6b25-4297-913b-e9bb1152952b None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:18.346622 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 154/299] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:19:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:18.347945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9dda1-5f47-420a-8c40-9428f7dea956 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:18.348225 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9dda1-5f47-420a-8c40-9428f7dea956 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:18.348498 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9dda1-5f47-420a-8c40-9428f7dea956 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:18.350433 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 155/300] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:18.355099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9dda1-5f47-420a-8c40-9428f7dea956 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:18.386757 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a0e5e-51b3-4808-b872-ff609f33bc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:18.387456 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a0e5e-51b3-4808-b872-ff609f33bc0b None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:18.387666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a0e5e-51b3-4808-b872-ff609f33bc0b None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:18.387940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a0e5e-51b3-4808-b872-ff609f33bc0b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:18.389671 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a0e5e-51b3-4808-b872-ff609f33bc0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:18.391172 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a0e5e-51b3-4808-b872-ff609f33bc0b None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:18.393217 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 146/301] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:18.394399 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f8e15-72c3-4040-8d0b-231678ed6137 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:18.394801 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f8e15-72c3-4040-8d0b-231678ed6137 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:18.394982 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f8e15-72c3-4040-8d0b-231678ed6137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:18.416387 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-9c0f8e15-72c3-4040-8d0b-231678ed6137 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:18.417746 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-9c0f8e15-72c3-4040-8d0b-231678ed6137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:18.473656 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9dda1-5f47-420a-8c40-9428f7dea956 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:18.490623 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 147/302] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:18.491701 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f8e15-72c3-4040-8d0b-231678ed6137 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:18.522281 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330192d1-cd6d-4835-abcf-926a5e5abb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:18.523305 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330192d1-cd6d-4835-abcf-926a5e5abb30 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:18.523795 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330192d1-cd6d-4835-abcf-926a5e5abb30 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:18.524375 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330192d1-cd6d-4835-abcf-926a5e5abb30 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:18.525387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330192d1-cd6d-4835-abcf-926a5e5abb30 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:18.525880 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330192d1-cd6d-4835-abcf-926a5e5abb30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:18.527469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330192d1-cd6d-4835-abcf-926a5e5abb30 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:18.537608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 156/303] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 10:19:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:18.545966 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330192d1-cd6d-4835-abcf-926a5e5abb30 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:18.549302 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:18.574357 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:18.618891 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f8e15-72c3-4040-8d0b-231678ed6137 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:18.656122 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762baf1-f385-4888-bf02-9376630ef639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:18.656898 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762baf1-f385-4888-bf02-9376630ef639 None admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:18.657240 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762baf1-f385-4888-bf02-9376630ef639 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:18.657669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762baf1-f385-4888-bf02-9376630ef639 None admin] PATH_INFO: `/v3/groups` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:18.658348 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762baf1-f385-4888-bf02-9376630ef639 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:18.659856 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762baf1-f385-4888-bf02-9376630ef639 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:18.660146 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330192d1-cd6d-4835-abcf-926a5e5abb30 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:18.679802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 148/304] 10.222.0.49 () {60 vars in 1134 bytes} [Mon Sep 26 10:19:18 2022] POST /identity/v3/groups => generated 229 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:19:18.705236 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6e829-ffc3-421d-860e-4aa3fe3b0362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:18.706232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6e829-ffc3-421d-860e-4aa3fe3b0362 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:18.706740 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6e829-ffc3-421d-860e-4aa3fe3b0362 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:18.707159 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6e829-ffc3-421d-860e-4aa3fe3b0362 None admin] PATH_INFO: `/v3/users/1d17cccbd72547a8892783dc7a1a74b9` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:18.724916 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a6e829-ffc3-421d-860e-4aa3fe3b0362 None admin] RBAC: Authorizing `identity:get_user(user_id=1d17cccbd72547a8892783dc7a1a74b9)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:18.726678 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a6e829-ffc3-421d-860e-4aa3fe3b0362 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:18.729049 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 157/305] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:19:18 2022] GET /identity/v3/users/1d17cccbd72547a8892783dc7a1a74b9 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:18.735425 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762baf1-f385-4888-bf02-9376630ef639 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:18.786786 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6e829-ffc3-421d-860e-4aa3fe3b0362 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:18.857758 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762baf1-f385-4888-bf02-9376630ef639 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:18.861015 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:18.901185 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c5e350-f1ae-4971-91c0-cc5a154fcb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:18.901803 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5e350-f1ae-4971-91c0-cc5a154fcb06 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:18.902122 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5e350-f1ae-4971-91c0-cc5a154fcb06 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:18.902404 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5e350-f1ae-4971-91c0-cc5a154fcb06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:18.903196 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c5e350-f1ae-4971-91c0-cc5a154fcb06 None admin] RBAC: Adding query filter params (role.id=4ffa6e25be8941078cca7449c84e50d5, user.id=1d17cccbd72547a8892783dc7a1a74b9, scope.system=all) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:18.903508 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c5e350-f1ae-4971-91c0-cc5a154fcb06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:18.905551 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c5e350-f1ae-4971-91c0-cc5a154fcb06 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:18.911353 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 149/306] 10.222.0.49 () {56 vars in 1308 bytes} [Mon Sep 26 10:19:18 2022] GET /identity/v3/role_assignments?user.id=1d17cccbd72547a8892783dc7a1a74b9&scope.system=all&role.id=4ffa6e25be8941078cca7449c84e50d5 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:18.919447 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:18.939904 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6e829-ffc3-421d-860e-4aa3fe3b0362 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:18.980611 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a79dec-ece9-47b6-af3c-b3fa0a6adb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:18.981255 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a79dec-ece9-47b6-af3c-b3fa0a6adb78 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:18.981649 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a79dec-ece9-47b6-af3c-b3fa0a6adb78 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:18.981958 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a79dec-ece9-47b6-af3c-b3fa0a6adb78 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:18.994978 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a79dec-ece9-47b6-af3c-b3fa0a6adb78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:19.000040 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-c0a79dec-ece9-47b6-af3c-b3fa0a6adb78 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:19:19.000040 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:19.000040 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:19.000040 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:19.000040 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:19.000040 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:19.000040 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:19.000040 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:19.000040 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:19.000040 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:19.000040 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:19.000040 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:19.000040 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:19.000040 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:19.000040 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:19.000040 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:19.000961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:19.000961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:19.000961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:19.000961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:19.000961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:19.000961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:19.000961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:19.000961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:19.000961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:19.000961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:19.000961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:19.000961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:19.000961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:19.000961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:19.000961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:19.000961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:19.001946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:19.001946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:19.001946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:19.001946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:19.001946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:19.001946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:19.001946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:19.001946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:19.001946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:19.001946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:19.001946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:19:19.001946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:19.001946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 158/307] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:19:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:19:19.007149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c5e350-f1ae-4971-91c0-cc5a154fcb06 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:19.121464 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c5e350-f1ae-4971-91c0-cc5a154fcb06 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:19.158107 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c164fe-b6d7-479b-ad10-7ea63d852254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:19.159123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c164fe-b6d7-479b-ad10-7ea63d852254 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:19.159601 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c164fe-b6d7-479b-ad10-7ea63d852254 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:19.160093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c164fe-b6d7-479b-ad10-7ea63d852254 None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:19.160954 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c164fe-b6d7-479b-ad10-7ea63d852254 None admin] RBAC: Adding query filter params (name=member) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:19.161508 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c164fe-b6d7-479b-ad10-7ea63d852254 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:19.163123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c164fe-b6d7-479b-ad10-7ea63d852254 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:19.172379 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 150/308] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 10:19:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:19.177575 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a79dec-ece9-47b6-af3c-b3fa0a6adb78 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:19.290643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a79dec-ece9-47b6-af3c-b3fa0a6adb78 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:19.327838 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6c2c1-c0f3-4fe8-a69a-ce675d3775fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:19.328793 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6c2c1-c0f3-4fe8-a69a-ce675d3775fe None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:19.329270 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6c2c1-c0f3-4fe8-a69a-ce675d3775fe None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:19.330076 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6c2c1-c0f3-4fe8-a69a-ce675d3775fe None admin] PATH_INFO: `/v3/users/6eebdc2cfd194fdf8f8f72087e212dd1` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:19.331452 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6c2c1-c0f3-4fe8-a69a-ce675d3775fe None admin] RBAC: Authorizing `identity:get_user(user_id=6eebdc2cfd194fdf8f8f72087e212dd1)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:19.333143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6c2c1-c0f3-4fe8-a69a-ce675d3775fe None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:19.335289 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 159/309] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:19:19 2022] GET /identity/v3/users/6eebdc2cfd194fdf8f8f72087e212dd1 => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:19.343091 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c164fe-b6d7-479b-ad10-7ea63d852254 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:19.375701 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:19.453381 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c164fe-b6d7-479b-ad10-7ea63d852254 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:19.491556 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1e7b3-927a-48b9-be05-b6d8d5be2a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:19.493005 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1e7b3-927a-48b9-be05-b6d8d5be2a71 None admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:19.493609 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1e7b3-927a-48b9-be05-b6d8d5be2a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:19.494137 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1e7b3-927a-48b9-be05-b6d8d5be2a71 None admin] PATH_INFO: `/v3/system/users/6eebdc2cfd194fdf8f8f72087e212dd1/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:19.495805 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac1e7b3-927a-48b9-be05-b6d8d5be2a71 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6eebdc2cfd194fdf8f8f72087e212dd1, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:19.497899 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac1e7b3-927a-48b9-be05-b6d8d5be2a71 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:19.510188 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 151/310] 10.222.0.49 () {58 vars in 1334 bytes} [Mon Sep 26 10:19:19 2022] PUT /identity/v3/system/users/6eebdc2cfd194fdf8f8f72087e212dd1/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:19:20.039567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c14cd7-67f2-4bb5-a5c5-ca618f82dded None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:20.039937 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c14cd7-67f2-4bb5-a5c5-ca618f82dded None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:20.040109 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c14cd7-67f2-4bb5-a5c5-ca618f82dded None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:20.041672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 160/311] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:20.047140 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb21c0-4e53-48d6-8dd9-b62cfdf208d4 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:20.047420 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb21c0-4e53-48d6-8dd9-b62cfdf208d4 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:20.047692 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb21c0-4e53-48d6-8dd9-b62cfdf208d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:20.067030 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-56fb21c0-4e53-48d6-8dd9-b62cfdf208d4 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:20.068134 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-56fb21c0-4e53-48d6-8dd9-b62cfdf208d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:20.138905 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 152/312] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 10:19:20.409941 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c14cd7-67f2-4bb5-a5c5-ca618f82dded None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:20.426223 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:20.514671 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c14cd7-67f2-4bb5-a5c5-ca618f82dded None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:20.549357 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1967cc-d79c-44af-bd2f-325d1d32755b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:20.550141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1967cc-d79c-44af-bd2f-325d1d32755b None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:20.550420 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1967cc-d79c-44af-bd2f-325d1d32755b None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:20.550687 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1967cc-d79c-44af-bd2f-325d1d32755b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:20.561565 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1967cc-d79c-44af-bd2f-325d1d32755b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:20.566643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-cb1967cc-d79c-44af-bd2f-325d1d32755b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 10:19:20.566643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:20.566643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:20.566643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:20.566643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:20.566643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:20.566643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:20.566643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:20.566643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:20.566643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:20.566643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:20.566643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:20.566643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:20.566643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:20.566643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:20.566643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:20.567657 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:20.567657 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:20.567657 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:20.567657 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:20.567657 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:20.567657 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:20.567657 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:20.567657 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:20.567657 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:20.567657 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:20.567657 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:20.567657 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:20.567657 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:20.567657 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:20.567657 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:20.567657 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:20.568935 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:20.568935 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:20.568935 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:20.568935 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:20.568935 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:20.568935 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:20.568935 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:20.568935 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:20.568935 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:20.568935 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:20.568935 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 10:19:20.568935 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:20.571966 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 161/313] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:19:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:20.578667 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb21c0-4e53-48d6-8dd9-b62cfdf208d4 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:20.604834 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:20.687429 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb21c0-4e53-48d6-8dd9-b62cfdf208d4 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:20.724042 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f2282-5dce-49a6-b119-08359b27803e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:20.724952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f2282-5dce-49a6-b119-08359b27803e None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:20.725431 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f2282-5dce-49a6-b119-08359b27803e None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:20.725878 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f2282-5dce-49a6-b119-08359b27803e None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:20.726732 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05f2282-5dce-49a6-b119-08359b27803e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:20.727207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05f2282-5dce-49a6-b119-08359b27803e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:20.728759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05f2282-5dce-49a6-b119-08359b27803e None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:20.737611 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 153/314] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 10:19:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:20.743308 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1967cc-d79c-44af-bd2f-325d1d32755b None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:20.831040 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:20.857311 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1967cc-d79c-44af-bd2f-325d1d32755b None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:20.897680 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a218d7-f43b-406a-9b66-510c48aeab1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:20.898584 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a218d7-f43b-406a-9b66-510c48aeab1d None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:20.898858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a218d7-f43b-406a-9b66-510c48aeab1d None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:20.899130 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a218d7-f43b-406a-9b66-510c48aeab1d None admin] PATH_INFO: `/v3/users/15dd612d16de458186fea2609c7fdf41` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:20.900276 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a218d7-f43b-406a-9b66-510c48aeab1d None admin] RBAC: Authorizing `identity:get_user(user_id=15dd612d16de458186fea2609c7fdf41)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:20.902044 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a218d7-f43b-406a-9b66-510c48aeab1d None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:20.904037 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 162/315] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:19:20 2022] GET /identity/v3/users/15dd612d16de458186fea2609c7fdf41 => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:20.911113 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f2282-5dce-49a6-b119-08359b27803e None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:20.981737 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14317fec-2441-4f41-b5b2-fc4a58300e34 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:20.981948 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14317fec-2441-4f41-b5b2-fc4a58300e34 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:20.982282 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14317fec-2441-4f41-b5b2-fc4a58300e34 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:20.983732 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 163/316] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:19:20.988534 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f6f06-1a54-43c8-b0ff-a8e4e6390adf None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:20.988809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f6f06-1a54-43c8-b0ff-a8e4e6390adf None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:20.989081 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f6f06-1a54-43c8-b0ff-a8e4e6390adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:21.007266 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:21.008372 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-107f6f06-1a54-43c8-b0ff-a8e4e6390adf None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:21.009560 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-107f6f06-1a54-43c8-b0ff-a8e4e6390adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:21.024414 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f2282-5dce-49a6-b119-08359b27803e None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:21.059566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1586bca-8c1d-4c6f-b923-45785365abcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:21.060897 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1586bca-8c1d-4c6f-b923-45785365abcb None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:21.061427 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1586bca-8c1d-4c6f-b923-45785365abcb None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:21.061887 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1586bca-8c1d-4c6f-b923-45785365abcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:21.084479 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 164/317] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 10:19:21.148380 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1586bca-8c1d-4c6f-b923-45785365abcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:21.150265 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1586bca-8c1d-4c6f-b923-45785365abcb None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:21.240211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:21.257697 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 154/318] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:19:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 10:19:21.263396 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1586bca-8c1d-4c6f-b923-45785365abcb None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:21.263875 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1669de6-59d6-4f1a-9263-a8503d79b070 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:21.264406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1669de6-59d6-4f1a-9263-a8503d79b070 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:21.264925 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1669de6-59d6-4f1a-9263-a8503d79b070 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:21.271638 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 165/319] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:21.274179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274250c1-d577-41df-9fff-c5cddd28482f None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:21.274672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274250c1-d577-41df-9fff-c5cddd28482f None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:21.275406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274250c1-d577-41df-9fff-c5cddd28482f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:21.296705 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-274250c1-d577-41df-9fff-c5cddd28482f None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:21.298469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-274250c1-d577-41df-9fff-c5cddd28482f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:21.375182 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 166/320] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 10:19:21.379545 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1586bca-8c1d-4c6f-b923-45785365abcb None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:21.382956 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274250c1-d577-41df-9fff-c5cddd28482f None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:21.385315 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:21.422062 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1b1fb-32ea-4ba3-ac06-ff86beea3bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:21.422751 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1b1fb-32ea-4ba3-ac06-ff86beea3bdf None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:21.423323 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1b1fb-32ea-4ba3-ac06-ff86beea3bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:21.423699 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1b1fb-32ea-4ba3-ac06-ff86beea3bdf None admin] PATH_INFO: `/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:21.425047 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c1b1fb-32ea-4ba3-ac06-ff86beea3bdf None admin] RBAC: Authorizing `identity:get_project(project_id=c3a5c629197a4f8ba3a3d1ad27faaa7e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:21.426706 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c1b1fb-32ea-4ba3-ac06-ff86beea3bdf None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:21.434300 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 155/321] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:19:21 2022] GET /identity/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:21.435772 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1b1fb-32ea-4ba3-ac06-ff86beea3bdf None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:21.503107 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274250c1-d577-41df-9fff-c5cddd28482f None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:21.504330 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:21.549504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec701075-d773-4d83-a655-b2cc74e5510b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:21.550599 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec701075-d773-4d83-a655-b2cc74e5510b None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:21.551133 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec701075-d773-4d83-a655-b2cc74e5510b None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:21.551592 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec701075-d773-4d83-a655-b2cc74e5510b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:21.553139 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec701075-d773-4d83-a655-b2cc74e5510b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:21.555469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec701075-d773-4d83-a655-b2cc74e5510b None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:21.561790 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1b1fb-32ea-4ba3-ac06-ff86beea3bdf None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:21.562450 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 167/322] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:21.564250 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec701075-d773-4d83-a655-b2cc74e5510b None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:21.604882 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958f13e-43d1-4965-b2df-faff6468a32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:21.605567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958f13e-43d1-4965-b2df-faff6468a32e None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:21.606658 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958f13e-43d1-4965-b2df-faff6468a32e None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:21.606658 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958f13e-43d1-4965-b2df-faff6468a32e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:21.608030 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b958f13e-43d1-4965-b2df-faff6468a32e None admin] RBAC: Adding query filter params (role.id=4ffa6e25be8941078cca7449c84e50d5, scope.project.id=c3a5c629197a4f8ba3a3d1ad27faaa7e, user.id=15dd612d16de458186fea2609c7fdf41) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:21.608337 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b958f13e-43d1-4965-b2df-faff6468a32e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:21.609844 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b958f13e-43d1-4965-b2df-faff6468a32e None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:21.630537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 156/323] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:19:21 2022] GET /identity/v3/role_assignments?user.id=15dd612d16de458186fea2609c7fdf41&scope.project.id=c3a5c629197a4f8ba3a3d1ad27faaa7e&role.id=4ffa6e25be8941078cca7449c84e50d5 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:21.649031 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958f13e-43d1-4965-b2df-faff6468a32e None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:21.658521 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:21.697893 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec701075-d773-4d83-a655-b2cc74e5510b None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:21.740648 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14096c44-ca75-4351-b116-0893480c734d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:21.741271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14096c44-ca75-4351-b116-0893480c734d None admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:21.741546 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14096c44-ca75-4351-b116-0893480c734d None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:21.741812 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14096c44-ca75-4351-b116-0893480c734d None admin] PATH_INFO: `/v3/groups` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:21.742484 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14096c44-ca75-4351-b116-0893480c734d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:21.743952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14096c44-ca75-4351-b116-0893480c734d None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:21.762534 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 168/324] 10.222.0.49 () {60 vars in 1134 bytes} [Mon Sep 26 10:19:21 2022] POST /identity/v3/groups => generated 226 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:19:21.764364 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb5aae-d536-4d56-b987-ec325c74e473 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:21.764646 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb5aae-d536-4d56-b987-ec325c74e473 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:21.764915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb5aae-d536-4d56-b987-ec325c74e473 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:21.770598 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 169/325] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:21.772306 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175a04d-31e9-4749-96b8-16c4fc256470 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:21.772591 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175a04d-31e9-4749-96b8-16c4fc256470 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:21.772860 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175a04d-31e9-4749-96b8-16c4fc256470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:21.795712 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-8175a04d-31e9-4749-96b8-16c4fc256470 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:21.797072 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-8175a04d-31e9-4749-96b8-16c4fc256470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:21.797697 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958f13e-43d1-4965-b2df-faff6468a32e None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:21.831220 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:21.860751 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2f965-0d76-4cd7-998c-4b323863f6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:21.860751 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2f965-0d76-4cd7-998c-4b323863f6c2 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:21.860751 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2f965-0d76-4cd7-998c-4b323863f6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:21.860751 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2f965-0d76-4cd7-998c-4b323863f6c2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:21.872585 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2f965-0d76-4cd7-998c-4b323863f6c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:21.879117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-e7a2f965-0d76-4cd7-998c-4b323863f6c2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 10:19:21.879117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:21.879117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:21.879117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:21.879117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:21.879117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:21.879117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:21.879117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:21.879117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:21.879117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:21.879117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:21.879117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:21.879117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:21.879117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:21.879117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:21.879117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:21.880073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:21.880073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:21.880073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:21.880073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:21.880073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:21.880073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:21.880073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:21.880073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:21.880073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:21.880073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:21.880073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:21.880073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:21.880073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:21.880073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:21.880073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:21.880073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:21.880868 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:21.880868 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:21.880868 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:21.880868 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:21.880868 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:21.880868 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:21.880868 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:21.880868 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:21.880868 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:21.880868 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:21.880868 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 10:19:21.880868 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:21.880868 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 157/326] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:19:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:19:21.885904 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2f965-0d76-4cd7-998c-4b323863f6c2 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:21.896795 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 170/327] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:22.001597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2f965-0d76-4cd7-998c-4b323863f6c2 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:22.035429 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f24a7-c1df-4c6a-b654-85952e78f55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:22.035967 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f24a7-c1df-4c6a-b654-85952e78f55f None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:22.036245 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f24a7-c1df-4c6a-b654-85952e78f55f None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:22.036517 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f24a7-c1df-4c6a-b654-85952e78f55f None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:22.037176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72f24a7-c1df-4c6a-b654-85952e78f55f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:22.037478 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72f24a7-c1df-4c6a-b654-85952e78f55f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:22.038895 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72f24a7-c1df-4c6a-b654-85952e78f55f None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:22.051695 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 158/328] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 10:19:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:22.053041 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175a04d-31e9-4749-96b8-16c4fc256470 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:22.123463 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:22.169507 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175a04d-31e9-4749-96b8-16c4fc256470 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:22.172485 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f24a7-c1df-4c6a-b654-85952e78f55f None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:22.216926 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad82032-6d72-4be9-9745-b7910749ae8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:22.217570 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad82032-6d72-4be9-9745-b7910749ae8c None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:22.217847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad82032-6d72-4be9-9745-b7910749ae8c None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:22.218154 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad82032-6d72-4be9-9745-b7910749ae8c None admin] PATH_INFO: `/v3/users/1d17cccbd72547a8892783dc7a1a74b9` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:22.219461 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad82032-6d72-4be9-9745-b7910749ae8c None admin] RBAC: Authorizing `identity:get_user(user_id=1d17cccbd72547a8892783dc7a1a74b9)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:22.220725 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad82032-6d72-4be9-9745-b7910749ae8c None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:22.224660 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 171/329] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:19:22 2022] GET /identity/v3/users/1d17cccbd72547a8892783dc7a1a74b9 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:22.229941 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad82032-6d72-4be9-9745-b7910749ae8c None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:22.234470 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:22.310717 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f24a7-c1df-4c6a-b654-85952e78f55f None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:22.355525 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad82032-6d72-4be9-9745-b7910749ae8c None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:22.362046 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b7914-084a-4058-ad0d-f4b0711fffc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:22.362046 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b7914-084a-4058-ad0d-f4b0711fffc2 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:22.362046 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b7914-084a-4058-ad0d-f4b0711fffc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:22.362046 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b7914-084a-4058-ad0d-f4b0711fffc2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:22.386205 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b7914-084a-4058-ad0d-f4b0711fffc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:22.393874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-bd3b7914-084a-4058-ad0d-f4b0711fffc2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:19:22.393874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:22.393874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:22.393874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:22.393874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:22.393874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:22.393874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:22.393874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:22.393874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:22.393874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:22.393874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:22.393874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:22.393874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:22.393874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:22.393874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:22.393874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:22.394820 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:22.394820 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:22.394820 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:22.394820 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:22.394820 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:22.394820 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:22.394820 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:22.394820 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:22.394820 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:22.394820 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:22.394820 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:22.394820 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:22.394820 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:22.394820 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:22.394820 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:22.394820 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:22.395618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:22.395618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:22.395618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:22.395618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:22.395618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:22.395618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:22.395618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:22.395618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:22.395618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:22.395618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:22.395618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:19:22.395618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:22.396443 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 159/330] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:19:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:19:22.396931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:22.401368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc81520-0ecd-4813-b252-1d39f5976ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:22.402047 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc81520-0ecd-4813-b252-1d39f5976ac3 None admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:22.402316 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b7914-084a-4058-ad0d-f4b0711fffc2 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:22.402420 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc81520-0ecd-4813-b252-1d39f5976ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:22.403307 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc81520-0ecd-4813-b252-1d39f5976ac3 None admin] PATH_INFO: `/v3/system/users/1d17cccbd72547a8892783dc7a1a74b9/roles/4ffa6e25be8941078cca7449c84e50d5` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:22.404781 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc81520-0ecd-4813-b252-1d39f5976ac3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1d17cccbd72547a8892783dc7a1a74b9, role_id=4ffa6e25be8941078cca7449c84e50d5)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:22.406677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc81520-0ecd-4813-b252-1d39f5976ac3 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:22.425856 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 172/331] 10.222.0.49 () {58 vars in 1334 bytes} [Mon Sep 26 10:19:22 2022] PUT /identity/v3/system/users/1d17cccbd72547a8892783dc7a1a74b9/roles/4ffa6e25be8941078cca7449c84e50d5 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:19:22.527264 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:22.530742 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b7914-084a-4058-ad0d-f4b0711fffc2 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:22.578507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81617792-a171-444f-8a31-1673e6d1b265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:22.579445 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81617792-a171-444f-8a31-1673e6d1b265 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:22.580159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81617792-a171-444f-8a31-1673e6d1b265 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:22.580600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81617792-a171-444f-8a31-1673e6d1b265 None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:22.581460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81617792-a171-444f-8a31-1673e6d1b265 None admin] RBAC: Adding query filter params (name=member) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:22.581939 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81617792-a171-444f-8a31-1673e6d1b265 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:22.583630 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81617792-a171-444f-8a31-1673e6d1b265 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:22.594655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 160/332] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 10:19:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:22.601004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc81520-0ecd-4813-b252-1d39f5976ac3 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:22.718695 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc81520-0ecd-4813-b252-1d39f5976ac3 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:22.758081 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7660e-6c15-41f4-85fa-ac55d77bff45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:22.758756 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7660e-6c15-41f4-85fa-ac55d77bff45 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:22.759036 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7660e-6c15-41f4-85fa-ac55d77bff45 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:22.759308 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7660e-6c15-41f4-85fa-ac55d77bff45 None admin] PATH_INFO: `/v3/users/6eebdc2cfd194fdf8f8f72087e212dd1` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:22.761079 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7660e-6c15-41f4-85fa-ac55d77bff45 None admin] RBAC: Authorizing `identity:get_user(user_id=6eebdc2cfd194fdf8f8f72087e212dd1)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:22.762635 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7660e-6c15-41f4-85fa-ac55d77bff45 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:22.764594 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 173/333] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:19:22 2022] GET /identity/v3/users/6eebdc2cfd194fdf8f8f72087e212dd1 => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:22.770210 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81617792-a171-444f-8a31-1673e6d1b265 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:22.938770 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81617792-a171-444f-8a31-1673e6d1b265 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:22.984004 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5eab5e-cb11-40da-bcaa-8960b4c013a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:22.984675 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5eab5e-cb11-40da-bcaa-8960b4c013a2 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:22.984952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5eab5e-cb11-40da-bcaa-8960b4c013a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:22.985219 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5eab5e-cb11-40da-bcaa-8960b4c013a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:22.985939 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5eab5e-cb11-40da-bcaa-8960b4c013a2 None admin] RBAC: Adding query filter params (role.id=dc7cfc688d354df4bc870fe8621c48bd, user.id=6eebdc2cfd194fdf8f8f72087e212dd1, scope.system=all) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:22.986269 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5eab5e-cb11-40da-bcaa-8960b4c013a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:22.987781 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5eab5e-cb11-40da-bcaa-8960b4c013a2 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:23.005258 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 161/334] 10.222.0.49 () {56 vars in 1308 bytes} [Mon Sep 26 10:19:22 2022] GET /identity/v3/role_assignments?user.id=6eebdc2cfd194fdf8f8f72087e212dd1&scope.system=all&role.id=dc7cfc688d354df4bc870fe8621c48bd => generated 510 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:24.547245 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587d2ba-b306-4dd5-8936-1a5196ef14bd None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:24.547762 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587d2ba-b306-4dd5-8936-1a5196ef14bd None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:24.547762 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587d2ba-b306-4dd5-8936-1a5196ef14bd None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:24.549273 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 174/335] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:24.555342 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f116106-7a4d-4f62-a7e9-b406c3d9a162 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:24.555774 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f116106-7a4d-4f62-a7e9-b406c3d9a162 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:24.555973 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f116106-7a4d-4f62-a7e9-b406c3d9a162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:24.576064 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-8f116106-7a4d-4f62-a7e9-b406c3d9a162 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:24.577278 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-8f116106-7a4d-4f62-a7e9-b406c3d9a162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:24.598756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:24.669166 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 162/336] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 10:19:25.024036 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae749dc1-dcb7-4791-b779-e3b36be650c8 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:25.024404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae749dc1-dcb7-4791-b779-e3b36be650c8 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:25.024599 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae749dc1-dcb7-4791-b779-e3b36be650c8 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:25.026218 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 175/337] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:25.032100 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704494d-9850-44d4-9378-0d9b3af0d3e4 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:25.032605 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704494d-9850-44d4-9378-0d9b3af0d3e4 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:25.033071 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704494d-9850-44d4-9378-0d9b3af0d3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:25.059164 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-6704494d-9850-44d4-9378-0d9b3af0d3e4 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:25.060349 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-6704494d-9850-44d4-9378-0d9b3af0d3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:25.138706 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 163/338] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:25.144983 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae749dc1-dcb7-4791-b779-e3b36be650c8 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:25.204660 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b94f838-b214-4b68-a643-a8a548a9a100 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:25.205196 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b94f838-b214-4b68-a643-a8a548a9a100 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:25.205471 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b94f838-b214-4b68-a643-a8a548a9a100 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:25.207098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 164/339] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:25.212463 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb65ead1-6c1d-4095-bc4a-74790a6f62ee None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:25.212807 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb65ead1-6c1d-4095-bc4a-74790a6f62ee None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:25.213178 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb65ead1-6c1d-4095-bc4a-74790a6f62ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:25.228069 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:25.232419 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-bb65ead1-6c1d-4095-bc4a-74790a6f62ee None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:25.233791 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-bb65ead1-6c1d-4095-bc4a-74790a6f62ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:25.309478 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae749dc1-dcb7-4791-b779-e3b36be650c8 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:25.340844 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 165/340] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 10:19:25.364721 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7258bd-d5b5-4a50-b949-226e23b97ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:25.365350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7258bd-d5b5-4a50-b949-226e23b97ede None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:25.365630 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7258bd-d5b5-4a50-b949-226e23b97ede None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:25.365902 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7258bd-d5b5-4a50-b949-226e23b97ede None admin] PATH_INFO: `/v3/roles/member` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:25.384048 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7258bd-d5b5-4a50-b949-226e23b97ede None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:25.389556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-ed7258bd-d5b5-4a50-b949-226e23b97ede None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:19:25.389556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:25.389556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:25.389556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:25.389556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:25.389556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:25.389556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:25.389556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:25.389556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:25.389556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:25.389556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:25.389556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:25.389556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:25.389556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:25.389556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:25.389556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:25.390430 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:25.390430 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:25.390430 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:25.390430 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:25.390430 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:25.390430 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:25.390430 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:25.390430 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:25.390430 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:25.390430 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:25.390430 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:25.390430 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:25.390430 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:25.390430 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:25.390430 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:25.390430 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:25.391237 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:25.391237 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:25.391237 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:25.391237 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:25.391237 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:25.391237 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:25.391237 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:25.391237 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:25.391237 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:25.391237 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:25.391237 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:19:25.391237 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:25.391237 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 176/341] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:19:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:25.397026 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb65ead1-6c1d-4095-bc4a-74790a6f62ee None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:25.413997 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:25.485769 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7cab4f-537d-4217-b46b-4283250412a1 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:25.486141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7cab4f-537d-4217-b46b-4283250412a1 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:25.486421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7cab4f-537d-4217-b46b-4283250412a1 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:25.487821 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 177/342] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:19:25.493421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa4082-6584-4d2e-9e55-4850b70a8b9e None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:25.493703 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa4082-6584-4d2e-9e55-4850b70a8b9e None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:25.493973 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa4082-6584-4d2e-9e55-4850b70a8b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:25.524380 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-ecfa4082-6584-4d2e-9e55-4850b70a8b9e None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:25.525595 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-ecfa4082-6584-4d2e-9e55-4850b70a8b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:25.556437 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb65ead1-6c1d-4095-bc4a-74790a6f62ee None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:25.602716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785dcd0e-d0f8-4cc5-895b-021ab4ecb92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:25.603259 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785dcd0e-d0f8-4cc5-895b-021ab4ecb92a None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:25.603528 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785dcd0e-d0f8-4cc5-895b-021ab4ecb92a None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:25.603803 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785dcd0e-d0f8-4cc5-895b-021ab4ecb92a None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:25.604479 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785dcd0e-d0f8-4cc5-895b-021ab4ecb92a None admin] RBAC: Adding query filter params (name=member) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:25.604779 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785dcd0e-d0f8-4cc5-895b-021ab4ecb92a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:25.608742 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785dcd0e-d0f8-4cc5-895b-021ab4ecb92a None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:25.618393 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 166/343] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 10:19:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:25.620225 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785dcd0e-d0f8-4cc5-895b-021ab4ecb92a None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:25.697087 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 178/344] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 10:19:25.698216 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa4082-6584-4d2e-9e55-4850b70a8b9e None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:25.760091 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:25.764232 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:25.831868 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785dcd0e-d0f8-4cc5-895b-021ab4ecb92a None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:25.850742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa4082-6584-4d2e-9e55-4850b70a8b9e None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:25.904700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c1cb7-ffc1-479a-95b3-c09a16aae0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:25.905266 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c1cb7-ffc1-479a-95b3-c09a16aae0f8 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:25.905544 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c1cb7-ffc1-479a-95b3-c09a16aae0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:25.905819 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c1cb7-ffc1-479a-95b3-c09a16aae0f8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:25.912029 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b0fa1a-898c-4f35-8097-ad64ba92fffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:25.912830 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b0fa1a-898c-4f35-8097-ad64ba92fffd None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:25.913250 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b0fa1a-898c-4f35-8097-ad64ba92fffd None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:25.913662 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b0fa1a-898c-4f35-8097-ad64ba92fffd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:25.925218 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412c1cb7-ffc1-479a-95b3-c09a16aae0f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:25.931894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-412c1cb7-ffc1-479a-95b3-c09a16aae0f8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 10:19:25.931894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:25.931894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:25.931894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:25.931894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:25.931894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:25.931894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:25.931894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:25.931894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:25.931894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:25.931894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:25.931894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:25.931894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:25.931894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:25.931894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:25.931894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:25.932963 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:25.932963 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:25.932963 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:25.932963 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:25.932963 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:25.932963 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:25.932963 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:25.932963 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:25.932963 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:25.932963 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:25.932963 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:25.932963 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:25.932963 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:25.932963 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:25.932963 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:25.932963 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:25.933962 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:25.933962 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:25.933962 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:25.933962 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:25.933962 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:25.933962 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:25.933962 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:25.933962 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:25.933962 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:25.933962 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:25.933962 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 10:19:25.933962 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:25.939567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 167/345] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:19:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:25.940519 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c1cb7-ffc1-479a-95b3-c09a16aae0f8 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:26.049826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b0fa1a-898c-4f35-8097-ad64ba92fffd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:26.051547 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b0fa1a-898c-4f35-8097-ad64ba92fffd None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:26.112765 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:26.118772 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c1cb7-ffc1-479a-95b3-c09a16aae0f8 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:26.155056 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:26.161172 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12df6b3-f785-4ac6-8554-3266929f9150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:26.161829 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12df6b3-f785-4ac6-8554-3266929f9150 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:26.162581 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12df6b3-f785-4ac6-8554-3266929f9150 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:26.162812 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12df6b3-f785-4ac6-8554-3266929f9150 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:26.172952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 179/346] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:19:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 10:19:26.174340 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b0fa1a-898c-4f35-8097-ad64ba92fffd None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:26.178079 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12df6b3-f785-4ac6-8554-3266929f9150 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:26.183136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-d12df6b3-f785-4ac6-8554-3266929f9150 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 10:19:26.183136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:26.183136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:26.183136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:26.183136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:26.183136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:26.183136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:26.183136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:26.183136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:26.183136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:26.183136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:26.183136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:26.183136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:26.183136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:26.183136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:26.183136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:26.184159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:26.184159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:26.184159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:26.184159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:26.184159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:26.184159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:26.184159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:26.184159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:26.184159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:26.184159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:26.184159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:26.184159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:26.184159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:26.184159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:26.184159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:26.184159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:26.184999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:26.184999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:26.184999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:26.184999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:26.184999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:26.184999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:26.184999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:26.184999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:26.184999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:26.184999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:26.184999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 10:19:26.184999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:26.184999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 168/347] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 10:19:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:26.185670 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12df6b3-f785-4ac6-8554-3266929f9150 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:26.294430 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b0fa1a-898c-4f35-8097-ad64ba92fffd None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:26.300591 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12df6b3-f785-4ac6-8554-3266929f9150 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:26.334345 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d1927-6cba-4280-9296-e2c5c844a194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:26.334959 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d1927-6cba-4280-9296-e2c5c844a194 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:26.335297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d1927-6cba-4280-9296-e2c5c844a194 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:26.335574 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d1927-6cba-4280-9296-e2c5c844a194 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:26.336240 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0d1927-6cba-4280-9296-e2c5c844a194 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:26.336544 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0d1927-6cba-4280-9296-e2c5c844a194 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:26.337912 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0d1927-6cba-4280-9296-e2c5c844a194 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:26.339770 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdebbfb-b3ae-4345-bc3b-f35c9bf2cc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:26.340585 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdebbfb-b3ae-4345-bc3b-f35c9bf2cc9f None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:26.340937 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdebbfb-b3ae-4345-bc3b-f35c9bf2cc9f None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:26.341915 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdebbfb-b3ae-4345-bc3b-f35c9bf2cc9f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:26.348988 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 180/348] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 10:19:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:26.349812 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d1927-6cba-4280-9296-e2c5c844a194 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:26.359205 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdebbfb-b3ae-4345-bc3b-f35c9bf2cc9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:26.364741 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:26.366239 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-5cdebbfb-b3ae-4345-bc3b-f35c9bf2cc9f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:19:26.366239 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:26.366239 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:26.366239 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:26.366239 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:26.366239 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:26.366239 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:26.366239 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:26.366239 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:26.366239 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:26.366239 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:26.366239 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:26.366239 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:26.366239 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:26.366239 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:26.366239 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:26.367144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:26.367144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:26.367144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:26.367144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:26.367144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:26.367144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:26.367144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:26.367144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:26.367144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:26.367144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:26.367144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:26.367144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:26.367144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:26.367144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:26.367144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:26.367144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:26.367930 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:26.367930 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:26.367930 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:26.367930 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:26.367930 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:26.367930 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:26.367930 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:26.367930 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:26.367930 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:26.367930 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:26.367930 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:19:26.367930 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:26.370493 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 169/349] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:19:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:26.371024 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdebbfb-b3ae-4345-bc3b-f35c9bf2cc9f None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:26.434910 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:26.471234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d1927-6cba-4280-9296-e2c5c844a194 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:26.493733 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdebbfb-b3ae-4345-bc3b-f35c9bf2cc9f None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:26.508885 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfbe4fb-4d3d-49e0-8da5-000af09ab693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:26.509913 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbe4fb-4d3d-49e0-8da5-000af09ab693 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:26.510230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbe4fb-4d3d-49e0-8da5-000af09ab693 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:26.510781 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbe4fb-4d3d-49e0-8da5-000af09ab693 None admin] PATH_INFO: `/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:26.512961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfbe4fb-4d3d-49e0-8da5-000af09ab693 None admin] RBAC: Authorizing `identity:get_project(project_id=f81f2060d8dd4c8a85edacc1547c6b3c)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:26.514988 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfbe4fb-4d3d-49e0-8da5-000af09ab693 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:26.516633 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 181/350] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:19:26 2022] GET /identity/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:26.517398 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfbe4fb-4d3d-49e0-8da5-000af09ab693 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:26.533838 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cecdca1-9d40-4fd8-b0a9-f47fad56a8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:26.534507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecdca1-9d40-4fd8-b0a9-f47fad56a8ad None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:26.534790 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecdca1-9d40-4fd8-b0a9-f47fad56a8ad None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:26.535062 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecdca1-9d40-4fd8-b0a9-f47fad56a8ad None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:26.535732 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cecdca1-9d40-4fd8-b0a9-f47fad56a8ad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:26.536034 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cecdca1-9d40-4fd8-b0a9-f47fad56a8ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:26.537427 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cecdca1-9d40-4fd8-b0a9-f47fad56a8ad None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:26.547668 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 170/351] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:19:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:26.548665 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cecdca1-9d40-4fd8-b0a9-f47fad56a8ad None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:26.624793 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:26.639884 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfbe4fb-4d3d-49e0-8da5-000af09ab693 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:26.669410 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cecdca1-9d40-4fd8-b0a9-f47fad56a8ad None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:26.678491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be1ebff-6de4-4018-829c-f5df6cd3f97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:26.679140 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1ebff-6de4-4018-829c-f5df6cd3f97f None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:26.679420 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1ebff-6de4-4018-829c-f5df6cd3f97f None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:26.679699 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1ebff-6de4-4018-829c-f5df6cd3f97f None admin] PATH_INFO: `/v3/users/1d17cccbd72547a8892783dc7a1a74b9` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:26.680850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be1ebff-6de4-4018-829c-f5df6cd3f97f None admin] RBAC: Authorizing `identity:get_user(user_id=1d17cccbd72547a8892783dc7a1a74b9)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:26.682328 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be1ebff-6de4-4018-829c-f5df6cd3f97f None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:26.684093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 182/352] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:19:26 2022] GET /identity/v3/users/1d17cccbd72547a8892783dc7a1a74b9 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:26.685431 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be1ebff-6de4-4018-829c-f5df6cd3f97f None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:26.689991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:26.714978 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c4ff7-af9a-4141-8349-8e8355f8bd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:26.716007 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c4ff7-af9a-4141-8349-8e8355f8bd4d None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:26.716966 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c4ff7-af9a-4141-8349-8e8355f8bd4d None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:26.717760 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c4ff7-af9a-4141-8349-8e8355f8bd4d None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:26.718947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c4ff7-af9a-4141-8349-8e8355f8bd4d None admin] RBAC: Adding query filter params (name=member) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:26.719952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c4ff7-af9a-4141-8349-8e8355f8bd4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:26.722129 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c4ff7-af9a-4141-8349-8e8355f8bd4d None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:26.733791 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 171/353] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 10:19:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:26.738634 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c4ff7-af9a-4141-8349-8e8355f8bd4d None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:26.808308 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be1ebff-6de4-4018-829c-f5df6cd3f97f None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:26.845127 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77839aa6-dad2-488b-b8c9-a08833062b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:26.845895 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77839aa6-dad2-488b-b8c9-a08833062b7b None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:26.846258 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77839aa6-dad2-488b-b8c9-a08833062b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:26.846594 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77839aa6-dad2-488b-b8c9-a08833062b7b None admin] PATH_INFO: `/v3/groups/f4ca40f956a2466f8aa17cf4de6b03ed` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:26.852750 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77839aa6-dad2-488b-b8c9-a08833062b7b None admin] RBAC: Authorizing `identity:get_group(group_id=f4ca40f956a2466f8aa17cf4de6b03ed)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:26.854250 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77839aa6-dad2-488b-b8c9-a08833062b7b None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:26.856030 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 183/354] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Sep 26 10:19:26 2022] GET /identity/v3/groups/f4ca40f956a2466f8aa17cf4de6b03ed => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:26.859601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77839aa6-dad2-488b-b8c9-a08833062b7b None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:26.862322 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c4ff7-af9a-4141-8349-8e8355f8bd4d None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:26.880733 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:26.902302 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625bb864-7651-457b-830a-941194d42443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:26.902922 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625bb864-7651-457b-830a-941194d42443 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:26.903201 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625bb864-7651-457b-830a-941194d42443 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:26.903475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625bb864-7651-457b-830a-941194d42443 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:26.967374 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:26.995223 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77839aa6-dad2-488b-b8c9-a08833062b7b None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:27.003432 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625bb864-7651-457b-830a-941194d42443 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:27.006972 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625bb864-7651-457b-830a-941194d42443 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:27.040420 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ecc27-252e-46b0-9bcf-dd7a02ada62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:27.041598 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ecc27-252e-46b0-9bcf-dd7a02ada62b None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:27.041598 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ecc27-252e-46b0-9bcf-dd7a02ada62b None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:27.041943 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ecc27-252e-46b0-9bcf-dd7a02ada62b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:27.043045 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ecc27-252e-46b0-9bcf-dd7a02ada62b None admin] RBAC: Adding query filter params (role.id=4ffa6e25be8941078cca7449c84e50d5, user.id=1d17cccbd72547a8892783dc7a1a74b9, scope.system=all) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:27.043355 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ecc27-252e-46b0-9bcf-dd7a02ada62b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:27.045319 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ecc27-252e-46b0-9bcf-dd7a02ada62b None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:27.059227 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 184/355] 10.222.0.49 () {56 vars in 1308 bytes} [Mon Sep 26 10:19:26 2022] GET /identity/v3/role_assignments?user.id=1d17cccbd72547a8892783dc7a1a74b9&scope.system=all&role.id=4ffa6e25be8941078cca7449c84e50d5 => generated 510 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:27.061211 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52abbc6-174f-4b98-b133-e986d4f43fc9 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:27.061665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52abbc6-174f-4b98-b133-e986d4f43fc9 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:27.061943 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52abbc6-174f-4b98-b133-e986d4f43fc9 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:27.063529 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 185/356] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:27.068802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52abbc6-174f-4b98-b133-e986d4f43fc9 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:27.139068 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 172/357] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:19:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 10:19:27.139947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625bb864-7651-457b-830a-941194d42443 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:27.161683 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:27.202396 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52abbc6-174f-4b98-b133-e986d4f43fc9 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:27.224617 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:27.245496 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ada0c-7865-4c32-a83f-89c0087de14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:27.246542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ada0c-7865-4c32-a83f-89c0087de14b None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:27.247024 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ada0c-7865-4c32-a83f-89c0087de14b None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:27.248096 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ada0c-7865-4c32-a83f-89c0087de14b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:27.255170 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625bb864-7651-457b-830a-941194d42443 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:27.306973 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b7f3d-4a95-43d4-a742-1f35ba52c346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:27.307609 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b7f3d-4a95-43d4-a742-1f35ba52c346 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:27.307864 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b7f3d-4a95-43d4-a742-1f35ba52c346 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:27.308135 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b7f3d-4a95-43d4-a742-1f35ba52c346 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:27.308874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b7f3d-4a95-43d4-a742-1f35ba52c346 None admin] RBAC: Adding query filter params (group.id=f4ca40f956a2466f8aa17cf4de6b03ed, role.id=dc7cfc688d354df4bc870fe8621c48bd, scope.project.id=f81f2060d8dd4c8a85edacc1547c6b3c) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:27.309180 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b7f3d-4a95-43d4-a742-1f35ba52c346 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:27.311260 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b7f3d-4a95-43d4-a742-1f35ba52c346 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:27.318202 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 173/358] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Sep 26 10:19:27 2022] GET /identity/v3/role_assignments?group.id=f4ca40f956a2466f8aa17cf4de6b03ed&scope.project.id=f81f2060d8dd4c8a85edacc1547c6b3c&role.id=dc7cfc688d354df4bc870fe8621c48bd => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:27.319502 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde7b29-3762-4682-a3dd-34b6b30a4ca2 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:27.319790 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde7b29-3762-4682-a3dd-34b6b30a4ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:27.320068 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde7b29-3762-4682-a3dd-34b6b30a4ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:27.339837 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ada0c-7865-4c32-a83f-89c0087de14b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:27.341727 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ada0c-7865-4c32-a83f-89c0087de14b None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:27.342829 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-0bde7b29-3762-4682-a3dd-34b6b30a4ca2 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:27.344092 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-0bde7b29-3762-4682-a3dd-34b6b30a4ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:27.436060 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 174/359] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:27.437229 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde7b29-3762-4682-a3dd-34b6b30a4ca2 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:27.440983 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:27.489299 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 186/360] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:19:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 10:19:27.497320 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b56d2-7926-451b-b59f-5a0b6bad120e None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:27.497943 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b56d2-7926-451b-b59f-5a0b6bad120e None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:27.498258 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b56d2-7926-451b-b59f-5a0b6bad120e None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:27.499886 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 187/361] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:27 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:27.501280 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b56d2-7926-451b-b59f-5a0b6bad120e None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:27.538254 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:27.578118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde7b29-3762-4682-a3dd-34b6b30a4ca2 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:27.622896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad161a-ab11-4032-8dd1-a1bc59e06dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:27.624024 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad161a-ab11-4032-8dd1-a1bc59e06dee None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:27.624392 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad161a-ab11-4032-8dd1-a1bc59e06dee None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:27.624740 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad161a-ab11-4032-8dd1-a1bc59e06dee None admin] PATH_INFO: `/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:27.626244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ad161a-ab11-4032-8dd1-a1bc59e06dee None admin] RBAC: Authorizing `identity:get_project(project_id=f81f2060d8dd4c8a85edacc1547c6b3c)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:27.628431 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ad161a-ab11-4032-8dd1-a1bc59e06dee None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:27.631012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 175/362] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:19:27 2022] GET /identity/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:27.632888 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa87efc-325b-49c9-a4ec-b77988147c56 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:27.633236 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa87efc-325b-49c9-a4ec-b77988147c56 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:27.633580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa87efc-325b-49c9-a4ec-b77988147c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:27.642175 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b56d2-7926-451b-b59f-5a0b6bad120e None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:27.660878 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-8fa87efc-325b-49c9-a4ec-b77988147c56 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:27.662509 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-8fa87efc-325b-49c9-a4ec-b77988147c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:27.687565 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c93bd2-8d12-4c1a-a58c-8bb74ba2b6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:27.688487 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c93bd2-8d12-4c1a-a58c-8bb74ba2b6ce None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:27.688971 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c93bd2-8d12-4c1a-a58c-8bb74ba2b6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:27.689425 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c93bd2-8d12-4c1a-a58c-8bb74ba2b6ce None admin] PATH_INFO: `/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:27.690774 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c93bd2-8d12-4c1a-a58c-8bb74ba2b6ce None admin] RBAC: Authorizing `identity:get_project(project_id=c3a5c629197a4f8ba3a3d1ad27faaa7e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:27.692476 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c93bd2-8d12-4c1a-a58c-8bb74ba2b6ce None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:27.694958 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 188/363] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:19:27 2022] GET /identity/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:27.696984 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c93bd2-8d12-4c1a-a58c-8bb74ba2b6ce None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:27.752208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:27.756915 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 176/364] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:27.759207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa87efc-325b-49c9-a4ec-b77988147c56 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:27.833245 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c93bd2-8d12-4c1a-a58c-8bb74ba2b6ce None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:27.854388 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:27.880321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0dab97-2375-4060-a9a0-d14b50c9ca49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:27.881186 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0dab97-2375-4060-a9a0-d14b50c9ca49 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:27.881635 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0dab97-2375-4060-a9a0-d14b50c9ca49 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:27.882068 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0dab97-2375-4060-a9a0-d14b50c9ca49 None admin] PATH_INFO: `/v3/groups/f4ca40f956a2466f8aa17cf4de6b03ed` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:27.883588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0dab97-2375-4060-a9a0-d14b50c9ca49 None admin] RBAC: Authorizing `identity:get_group(group_id=f4ca40f956a2466f8aa17cf4de6b03ed)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:27.885549 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0dab97-2375-4060-a9a0-d14b50c9ca49 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:27.887639 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 189/365] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Sep 26 10:19:27 2022] GET /identity/v3/groups/f4ca40f956a2466f8aa17cf4de6b03ed => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:27.889233 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0dab97-2375-4060-a9a0-d14b50c9ca49 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:27.901629 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa87efc-325b-49c9-a4ec-b77988147c56 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:27.943666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389afaa-f7b5-42f4-8e80-1cdc125e0b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:27.944692 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389afaa-f7b5-42f4-8e80-1cdc125e0b4d None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:27.945144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389afaa-f7b5-42f4-8e80-1cdc125e0b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:27.945682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389afaa-f7b5-42f4-8e80-1cdc125e0b4d None admin] PATH_INFO: `/v3/groups/f4ca40f956a2466f8aa17cf4de6b03ed` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:27.947108 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389afaa-f7b5-42f4-8e80-1cdc125e0b4d None admin] RBAC: Authorizing `identity:get_group(group_id=f4ca40f956a2466f8aa17cf4de6b03ed)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:27.948841 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389afaa-f7b5-42f4-8e80-1cdc125e0b4d None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:27.951229 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 177/366] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Sep 26 10:19:27 2022] GET /identity/v3/groups/f4ca40f956a2466f8aa17cf4de6b03ed => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:27.952158 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389afaa-f7b5-42f4-8e80-1cdc125e0b4d None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:28.032134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0dab97-2375-4060-a9a0-d14b50c9ca49 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:28.039889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:28.069931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389afaa-f7b5-42f4-8e80-1cdc125e0b4d None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:28.077185 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db87c4-2518-48df-acdc-b56b09d92ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:28.077837 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db87c4-2518-48df-acdc-b56b09d92ffa None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:28.078146 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db87c4-2518-48df-acdc-b56b09d92ffa None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:28.078422 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db87c4-2518-48df-acdc-b56b09d92ffa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:28.091653 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13db87c4-2518-48df-acdc-b56b09d92ffa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:28.096665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-13db87c4-2518-48df-acdc-b56b09d92ffa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 10:19:28.096665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:28.096665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:28.096665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:28.096665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:28.096665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:28.096665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:28.096665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:28.096665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:28.096665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:28.096665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:28.096665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:28.096665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:28.096665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:28.096665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:28.096665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:28.097585 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:28.097585 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:28.097585 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:28.097585 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:28.097585 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:28.097585 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:28.097585 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:28.097585 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:28.097585 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:28.097585 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:28.097585 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:28.097585 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:28.097585 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:28.097585 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:28.097585 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:28.097585 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:28.098435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:28.098435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:28.098435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:28.098435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:28.098435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:28.098435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:28.098435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:28.098435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:28.098435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:28.098435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:28.098435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 10:19:28.098435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:28.098435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 190/367] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 10:19:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:28.099096 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db87c4-2518-48df-acdc-b56b09d92ffa None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:28.117000 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638be000-c4b8-41ad-8bd3-8e19ac87d9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:28.117617 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638be000-c4b8-41ad-8bd3-8e19ac87d9ce None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:28.117897 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638be000-c4b8-41ad-8bd3-8e19ac87d9ce None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:28.118208 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638be000-c4b8-41ad-8bd3-8e19ac87d9ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:28.118946 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638be000-c4b8-41ad-8bd3-8e19ac87d9ce None admin] RBAC: Adding query filter params (group.id=f4ca40f956a2466f8aa17cf4de6b03ed, role.id=d1109db57b004dd9a4f84a634ff76205, scope.project.id=f81f2060d8dd4c8a85edacc1547c6b3c) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:28.119260 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638be000-c4b8-41ad-8bd3-8e19ac87d9ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:28.120701 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638be000-c4b8-41ad-8bd3-8e19ac87d9ce None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:28.127589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:28.134480 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 178/368] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Sep 26 10:19:27 2022] GET /identity/v3/role_assignments?group.id=f4ca40f956a2466f8aa17cf4de6b03ed&scope.project.id=f81f2060d8dd4c8a85edacc1547c6b3c&role.id=d1109db57b004dd9a4f84a634ff76205 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:28.135435 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638be000-c4b8-41ad-8bd3-8e19ac87d9ce None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:28.260262 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db87c4-2518-48df-acdc-b56b09d92ffa None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:28.294224 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638be000-c4b8-41ad-8bd3-8e19ac87d9ce None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:28.310390 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e77c5c-ed84-4353-b286-68bdc3ca1208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:28.311006 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e77c5c-ed84-4353-b286-68bdc3ca1208 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:28.311285 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e77c5c-ed84-4353-b286-68bdc3ca1208 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:28.311562 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e77c5c-ed84-4353-b286-68bdc3ca1208 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:28.312319 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e77c5c-ed84-4353-b286-68bdc3ca1208 None admin] RBAC: Adding query filter params (group.id=f4ca40f956a2466f8aa17cf4de6b03ed, role.id=dc7cfc688d354df4bc870fe8621c48bd, scope.project.id=c3a5c629197a4f8ba3a3d1ad27faaa7e) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:28.312616 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e77c5c-ed84-4353-b286-68bdc3ca1208 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:28.314691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e77c5c-ed84-4353-b286-68bdc3ca1208 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:28.326219 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 191/369] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Sep 26 10:19:28 2022] GET /identity/v3/role_assignments?group.id=f4ca40f956a2466f8aa17cf4de6b03ed&scope.project.id=c3a5c629197a4f8ba3a3d1ad27faaa7e&role.id=dc7cfc688d354df4bc870fe8621c48bd => generated 260 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:28.326219 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e77c5c-ed84-4353-b286-68bdc3ca1208 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:28.354431 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f189312d-6f8e-4930-aa9b-30ac38f8d1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:28.355531 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189312d-6f8e-4930-aa9b-30ac38f8d1db None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:28.356017 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189312d-6f8e-4930-aa9b-30ac38f8d1db None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:28.356496 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189312d-6f8e-4930-aa9b-30ac38f8d1db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:28.376651 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f189312d-6f8e-4930-aa9b-30ac38f8d1db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:28.387213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-f189312d-6f8e-4930-aa9b-30ac38f8d1db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:19:28.387213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:28.387213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:28.387213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:28.387213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:28.387213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:28.387213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:28.387213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:28.387213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:28.387213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:28.387213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:28.387213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:28.387213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:28.387213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:28.387213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:28.387213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:28.388226 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:28.388226 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:28.388226 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:28.388226 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:28.388226 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:28.388226 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:28.388226 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:28.388226 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:28.388226 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:28.388226 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:28.388226 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:28.388226 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:28.388226 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:28.388226 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:28.388226 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:28.388226 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:28.389037 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:28.389037 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:28.389037 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:28.389037 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:28.389037 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:28.389037 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:28.389037 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:28.389037 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:28.389037 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:28.389037 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:28.389037 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:19:28.389037 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:28.391877 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:28.392714 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 179/370] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:19:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:28.399086 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f189312d-6f8e-4930-aa9b-30ac38f8d1db None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:28.491495 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:28.494605 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e77c5c-ed84-4353-b286-68bdc3ca1208 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:28.544677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f189312d-6f8e-4930-aa9b-30ac38f8d1db None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:28.556632 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4968e08-cdb5-4a3c-bbd5-8f4ed8684757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:28.557548 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4968e08-cdb5-4a3c-bbd5-8f4ed8684757 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:28.557999 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4968e08-cdb5-4a3c-bbd5-8f4ed8684757 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:28.558458 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4968e08-cdb5-4a3c-bbd5-8f4ed8684757 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:28.559374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4968e08-cdb5-4a3c-bbd5-8f4ed8684757 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:28.559822 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4968e08-cdb5-4a3c-bbd5-8f4ed8684757 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:28.561391 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4968e08-cdb5-4a3c-bbd5-8f4ed8684757 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:28.581712 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 192/371] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:19:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:28.585751 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def6eec-c808-481f-8ca4-47e7cb7254f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:28.586630 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def6eec-c808-481f-8ca4-47e7cb7254f1 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:28.587012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def6eec-c808-481f-8ca4-47e7cb7254f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:28.587333 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4968e08-cdb5-4a3c-bbd5-8f4ed8684757 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:28.587445 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def6eec-c808-481f-8ca4-47e7cb7254f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:28.588205 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def6eec-c808-481f-8ca4-47e7cb7254f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:28.588933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def6eec-c808-481f-8ca4-47e7cb7254f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:28.590126 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def6eec-c808-481f-8ca4-47e7cb7254f1 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:28.606048 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 180/372] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:19:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:28.609769 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def6eec-c808-481f-8ca4-47e7cb7254f1 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:28.737485 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def6eec-c808-481f-8ca4-47e7cb7254f1 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:28.743068 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:28.760626 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4968e08-cdb5-4a3c-bbd5-8f4ed8684757 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:28.764640 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:28.778301 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d650fc5-0da7-408e-b4cc-3ccdf4d02c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:28.779011 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d650fc5-0da7-408e-b4cc-3ccdf4d02c54 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:28.779292 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d650fc5-0da7-408e-b4cc-3ccdf4d02c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:28.779567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d650fc5-0da7-408e-b4cc-3ccdf4d02c54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:28.822742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca990cd5-4709-4e1a-9583-97f862b15837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:28.823680 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca990cd5-4709-4e1a-9583-97f862b15837 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:28.824181 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca990cd5-4709-4e1a-9583-97f862b15837 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:28.824653 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca990cd5-4709-4e1a-9583-97f862b15837 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:28.862050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d650fc5-0da7-408e-b4cc-3ccdf4d02c54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:28.862050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d650fc5-0da7-408e-b4cc-3ccdf4d02c54 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:28.929900 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca990cd5-4709-4e1a-9583-97f862b15837 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:28.932076 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca990cd5-4709-4e1a-9583-97f862b15837 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:28.974059 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 181/373] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:19:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 10:19:28.975360 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d650fc5-0da7-408e-b4cc-3ccdf4d02c54 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:29.014400 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:29.090975 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d650fc5-0da7-408e-b4cc-3ccdf4d02c54 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:29.128625 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e806736-8034-4b77-84c0-1b4bbf1ae8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:29.129269 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e806736-8034-4b77-84c0-1b4bbf1ae8ca None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:29.129546 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e806736-8034-4b77-84c0-1b4bbf1ae8ca None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:29.129825 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e806736-8034-4b77-84c0-1b4bbf1ae8ca None admin] PATH_INFO: `/v3/projects/9cef67c15d1f4b9695af7c22c68b5822` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:29.131008 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e806736-8034-4b77-84c0-1b4bbf1ae8ca None admin] RBAC: Authorizing `identity:get_project(project_id=9cef67c15d1f4b9695af7c22c68b5822)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:29.132589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e806736-8034-4b77-84c0-1b4bbf1ae8ca None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:29.134695 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 182/374] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:19:28 2022] GET /identity/v3/projects/9cef67c15d1f4b9695af7c22c68b5822 => generated 344 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:29.146436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e806736-8034-4b77-84c0-1b4bbf1ae8ca None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:29.155473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:29.177811 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 193/375] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:19:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 10:19:29.180698 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca990cd5-4709-4e1a-9583-97f862b15837 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:29.316643 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:29.331726 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e806736-8034-4b77-84c0-1b4bbf1ae8ca None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:29.372953 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3caac19-e0fd-47c8-84be-1c9e7349eff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:29.373929 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3caac19-e0fd-47c8-84be-1c9e7349eff2 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:29.374431 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3caac19-e0fd-47c8-84be-1c9e7349eff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:29.375071 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3caac19-e0fd-47c8-84be-1c9e7349eff2 None admin] PATH_INFO: `/v3/groups/6960f263897a4e93ac28513cac7ff71e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:29.381357 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3caac19-e0fd-47c8-84be-1c9e7349eff2 None admin] RBAC: Authorizing `identity:get_group(group_id=6960f263897a4e93ac28513cac7ff71e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:29.383072 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3caac19-e0fd-47c8-84be-1c9e7349eff2 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:29.385422 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 183/376] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Sep 26 10:19:29 2022] GET /identity/v3/groups/6960f263897a4e93ac28513cac7ff71e => generated 229 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:29.387922 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca990cd5-4709-4e1a-9583-97f862b15837 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:29.400987 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3caac19-e0fd-47c8-84be-1c9e7349eff2 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:29.444175 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8826ae-a66b-4eb5-8d74-e20f40947625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:29.444814 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8826ae-a66b-4eb5-8d74-e20f40947625 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:29.445092 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8826ae-a66b-4eb5-8d74-e20f40947625 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:29.445364 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8826ae-a66b-4eb5-8d74-e20f40947625 None admin] PATH_INFO: `/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:29.447140 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8826ae-a66b-4eb5-8d74-e20f40947625 None admin] RBAC: Authorizing `identity:get_project(project_id=c3a5c629197a4f8ba3a3d1ad27faaa7e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:29.448607 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8826ae-a66b-4eb5-8d74-e20f40947625 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:29.450293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 194/377] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:19:29 2022] GET /identity/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e => generated 302 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:29.457309 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8826ae-a66b-4eb5-8d74-e20f40947625 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:29.544582 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:29.566125 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3caac19-e0fd-47c8-84be-1c9e7349eff2 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:29.610111 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d64592-5a90-415f-92a0-11362cb51f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:29.610707 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d64592-5a90-415f-92a0-11362cb51f82 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:29.611429 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d64592-5a90-415f-92a0-11362cb51f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:29.611703 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d64592-5a90-415f-92a0-11362cb51f82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:29.612709 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d64592-5a90-415f-92a0-11362cb51f82 None admin] RBAC: Adding query filter params (group.id=6960f263897a4e93ac28513cac7ff71e, role.id=532b37d9076c45618cf352aafb85f35e, scope.project.id=9cef67c15d1f4b9695af7c22c68b5822) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:29.613020 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d64592-5a90-415f-92a0-11362cb51f82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:29.614714 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d64592-5a90-415f-92a0-11362cb51f82 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:29.621258 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 184/378] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Sep 26 10:19:29 2022] GET /identity/v3/role_assignments?group.id=6960f263897a4e93ac28513cac7ff71e&scope.project.id=9cef67c15d1f4b9695af7c22c68b5822&role.id=532b37d9076c45618cf352aafb85f35e => generated 260 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:29.633336 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8826ae-a66b-4eb5-8d74-e20f40947625 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:29.705752 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060c6cfb-eeb4-4962-b008-cbe18dbc3f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:29.706521 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c6cfb-eeb4-4962-b008-cbe18dbc3f1a None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:29.706797 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c6cfb-eeb4-4962-b008-cbe18dbc3f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:29.707068 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c6cfb-eeb4-4962-b008-cbe18dbc3f1a None admin] PATH_INFO: `/v3/groups/f4ca40f956a2466f8aa17cf4de6b03ed` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:29.710774 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060c6cfb-eeb4-4962-b008-cbe18dbc3f1a None admin] RBAC: Authorizing `identity:get_group(group_id=f4ca40f956a2466f8aa17cf4de6b03ed)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:29.712267 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060c6cfb-eeb4-4962-b008-cbe18dbc3f1a None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:29.714298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 195/379] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Sep 26 10:19:29 2022] GET /identity/v3/groups/f4ca40f956a2466f8aa17cf4de6b03ed => generated 226 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:29.720377 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d64592-5a90-415f-92a0-11362cb51f82 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:29.758960 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:29.879098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d64592-5a90-415f-92a0-11362cb51f82 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:29.928855 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f448e7-f9d9-4f0f-bb7f-c02e158eeb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:29.929389 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f448e7-f9d9-4f0f-bb7f-c02e158eeb5f None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:29.929669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f448e7-f9d9-4f0f-bb7f-c02e158eeb5f None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:29.929942 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f448e7-f9d9-4f0f-bb7f-c02e158eeb5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:29.930698 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f448e7-f9d9-4f0f-bb7f-c02e158eeb5f None admin] RBAC: Adding query filter params (group.id=f4ca40f956a2466f8aa17cf4de6b03ed, role.id=d1109db57b004dd9a4f84a634ff76205, scope.project.id=c3a5c629197a4f8ba3a3d1ad27faaa7e) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:29.931011 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f448e7-f9d9-4f0f-bb7f-c02e158eeb5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:29.932465 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f448e7-f9d9-4f0f-bb7f-c02e158eeb5f None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:29.941264 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 185/380] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Sep 26 10:19:29 2022] GET /identity/v3/role_assignments?group.id=f4ca40f956a2466f8aa17cf4de6b03ed&scope.project.id=c3a5c629197a4f8ba3a3d1ad27faaa7e&role.id=d1109db57b004dd9a4f84a634ff76205 => generated 260 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:29.995054 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b1d03-cf9e-42e4-8757-b899e70ffb19 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:29.995730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b1d03-cf9e-42e4-8757-b899e70ffb19 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:29.995730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b1d03-cf9e-42e4-8757-b899e70ffb19 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:29.997124 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 196/381] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:30.003226 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ab9f8-bef2-4628-ac61-985a9b0d92fe None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:30.003505 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ab9f8-bef2-4628-ac61-985a9b0d92fe None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:30.003768 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ab9f8-bef2-4628-ac61-985a9b0d92fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:30.027189 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-d33ab9f8-bef2-4628-ac61-985a9b0d92fe None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:30.028424 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-d33ab9f8-bef2-4628-ac61-985a9b0d92fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:30.128567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 186/382] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:30.481636 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b1d03-cf9e-42e4-8757-b899e70ffb19 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:30.604581 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b1d03-cf9e-42e4-8757-b899e70ffb19 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:30.633825 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:30.643395 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a0e619-9dc3-4578-8f3c-983b02f5883c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:30.644035 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0e619-9dc3-4578-8f3c-983b02f5883c None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:30.644309 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0e619-9dc3-4578-8f3c-983b02f5883c None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:30.644594 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0e619-9dc3-4578-8f3c-983b02f5883c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:30.655359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a0e619-9dc3-4578-8f3c-983b02f5883c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:30.660407 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-69a0e619-9dc3-4578-8f3c-983b02f5883c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:19:30.660407 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:30.660407 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:30.660407 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:30.660407 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:30.660407 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:30.660407 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:30.660407 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:30.660407 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:30.660407 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:30.660407 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:30.660407 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:30.660407 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:30.660407 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:30.660407 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:30.660407 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:30.661270 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:30.661270 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:30.661270 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:30.661270 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:30.661270 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:30.661270 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:30.661270 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:30.661270 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:30.661270 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:30.661270 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:30.661270 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:30.661270 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:30.661270 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:30.661270 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:30.661270 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:30.661270 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:30.662231 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:30.662231 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:30.662231 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:30.662231 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:30.662231 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:30.662231 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:30.662231 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:30.662231 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:30.662231 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:30.662231 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:30.662231 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:19:30.662231 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:30.662231 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 197/383] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:19:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:30.667374 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ab9f8-bef2-4628-ac61-985a9b0d92fe None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:30.693374 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:30.815583 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbd712-b4fb-4b89-b231-c8d6538ecf7d None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:30.815883 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbd712-b4fb-4b89-b231-c8d6538ecf7d None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:30.816124 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbd712-b4fb-4b89-b231-c8d6538ecf7d None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:30.817474 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 198/384] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:19:30.822801 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c1926-433b-4b52-8aed-9831c1f033ad None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:30.823195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c1926-433b-4b52-8aed-9831c1f033ad None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:30.823585 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c1926-433b-4b52-8aed-9831c1f033ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:30.825128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ab9f8-bef2-4628-ac61-985a9b0d92fe None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:30.842696 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-985c1926-433b-4b52-8aed-9831c1f033ad None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:30.844032 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-985c1926-433b-4b52-8aed-9831c1f033ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:30.880401 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a914b6-7b16-48a8-a937-66bd77d8bfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:30.881044 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a914b6-7b16-48a8-a937-66bd77d8bfb1 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:30.881282 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a914b6-7b16-48a8-a937-66bd77d8bfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:30.881550 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a914b6-7b16-48a8-a937-66bd77d8bfb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:30.882235 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a914b6-7b16-48a8-a937-66bd77d8bfb1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:30.882535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a914b6-7b16-48a8-a937-66bd77d8bfb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:30.883989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a914b6-7b16-48a8-a937-66bd77d8bfb1 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:30.896307 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 187/385] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 10:19:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:30.902258 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a914b6-7b16-48a8-a937-66bd77d8bfb1 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:30.971906 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 199/386] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 10:19:31.016483 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:31.046619 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a914b6-7b16-48a8-a937-66bd77d8bfb1 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:31.087892 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cb17e-0adc-425a-ab27-d8c905d42858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:31.088441 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cb17e-0adc-425a-ab27-d8c905d42858 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:31.088714 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cb17e-0adc-425a-ab27-d8c905d42858 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:31.088985 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cb17e-0adc-425a-ab27-d8c905d42858 None admin] PATH_INFO: `/v3/groups/f4ca40f956a2466f8aa17cf4de6b03ed` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:31.090272 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42cb17e-0adc-425a-ab27-d8c905d42858 None admin] RBAC: Authorizing `identity:get_group(group_id=f4ca40f956a2466f8aa17cf4de6b03ed)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:31.091743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42cb17e-0adc-425a-ab27-d8c905d42858 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:31.093623 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 188/387] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Sep 26 10:19:30 2022] GET /identity/v3/groups/f4ca40f956a2466f8aa17cf4de6b03ed => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:31.098846 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985c1926-433b-4b52-8aed-9831c1f033ad None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:31.209929 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985c1926-433b-4b52-8aed-9831c1f033ad None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:31.234211 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:31.252967 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080be70e-d0b5-4169-94c1-9028549d9285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:31.253664 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080be70e-d0b5-4169-94c1-9028549d9285 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:31.253945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080be70e-d0b5-4169-94c1-9028549d9285 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:31.254281 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080be70e-d0b5-4169-94c1-9028549d9285 None admin] PATH_INFO: `/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:31.255972 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080be70e-d0b5-4169-94c1-9028549d9285 None admin] RBAC: Authorizing `identity:get_project(project_id=f81f2060d8dd4c8a85edacc1547c6b3c)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:31.257412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080be70e-d0b5-4169-94c1-9028549d9285 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:31.259251 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 200/388] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:19:31 2022] GET /identity/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:31.265017 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cb17e-0adc-425a-ab27-d8c905d42858 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:31.397513 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cb17e-0adc-425a-ab27-d8c905d42858 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:31.439455 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e66495-92f2-4834-9542-8794b41aeec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:31.440122 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e66495-92f2-4834-9542-8794b41aeec6 None admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:31.440396 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e66495-92f2-4834-9542-8794b41aeec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:31.440664 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e66495-92f2-4834-9542-8794b41aeec6 None admin] PATH_INFO: `/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c/groups/f4ca40f956a2466f8aa17cf4de6b03ed/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:31.442575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e66495-92f2-4834-9542-8794b41aeec6 None admin] RBAC: Authorizing `identity:create_grant(project_id=f81f2060d8dd4c8a85edacc1547c6b3c, group_id=f4ca40f956a2466f8aa17cf4de6b03ed, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:31.444001 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e66495-92f2-4834-9542-8794b41aeec6 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:31.472982 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 189/389] 10.222.0.49 () {58 vars in 1442 bytes} [Mon Sep 26 10:19:31 2022] PUT /identity/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c/groups/f4ca40f956a2466f8aa17cf4de6b03ed/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:19:31.606453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00aac8-9d07-4f5a-9736-6c63dc16104f None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:31.607280 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00aac8-9d07-4f5a-9736-6c63dc16104f None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:31.607769 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00aac8-9d07-4f5a-9736-6c63dc16104f None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:31.609501 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 201/390] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:31.615073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15fceb-a364-4c6f-b0be-6c7011dd0a8b None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:31.615351 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15fceb-a364-4c6f-b0be-6c7011dd0a8b None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:31.615633 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15fceb-a364-4c6f-b0be-6c7011dd0a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:31.622237 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00aac8-9d07-4f5a-9736-6c63dc16104f None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:31.625235 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:31.646213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-ce15fceb-a364-4c6f-b0be-6c7011dd0a8b None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:31.647732 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-ce15fceb-a364-4c6f-b0be-6c7011dd0a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:31.767288 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 190/391] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:31.797044 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00aac8-9d07-4f5a-9736-6c63dc16104f None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:31.838236 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6bb9b-7276-41e7-9cd0-8ccf1c8371c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:31.839126 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6bb9b-7276-41e7-9cd0-8ccf1c8371c9 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:31.839612 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6bb9b-7276-41e7-9cd0-8ccf1c8371c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:31.840025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6bb9b-7276-41e7-9cd0-8ccf1c8371c9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:31.855361 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d6bb9b-7276-41e7-9cd0-8ccf1c8371c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:31.860914 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-b4d6bb9b-7276-41e7-9cd0-8ccf1c8371c9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 10:19:31.860914 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:31.860914 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:31.860914 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:31.860914 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:31.860914 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:31.860914 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:31.860914 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:31.860914 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:31.860914 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:31.860914 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:31.860914 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:31.860914 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:31.860914 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:31.860914 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:31.860914 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:31.861840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:31.861840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:31.861840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:31.861840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:31.861840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:31.861840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:31.861840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:31.861840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:31.861840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:31.861840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:31.861840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:31.861840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:31.861840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:31.861840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:31.861840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:31.861840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:31.862713 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:31.862713 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:31.862713 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:31.862713 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:31.862713 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:31.862713 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:31.862713 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:31.862713 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:31.862713 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:31.862713 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:31.862713 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 10:19:31.862713 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:31.862713 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 202/392] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 10:19:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:19:31.869980 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce15fceb-a364-4c6f-b0be-6c7011dd0a8b None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:31.990571 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce15fceb-a364-4c6f-b0be-6c7011dd0a8b None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:32.023905 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:32.030635 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79880c64-142e-4c7c-ae7a-dd9b940746bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:32.031247 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79880c64-142e-4c7c-ae7a-dd9b940746bf None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:32.031527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79880c64-142e-4c7c-ae7a-dd9b940746bf None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:32.031800 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79880c64-142e-4c7c-ae7a-dd9b940746bf None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:32.032470 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79880c64-142e-4c7c-ae7a-dd9b940746bf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:32.032769 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79880c64-142e-4c7c-ae7a-dd9b940746bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:32.034246 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79880c64-142e-4c7c-ae7a-dd9b940746bf None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:32.045880 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 191/393] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:19:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:32.051989 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6bb9b-7276-41e7-9cd0-8ccf1c8371c9 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:32.113549 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:32.188768 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79880c64-142e-4c7c-ae7a-dd9b940746bf None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:32.210690 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6bb9b-7276-41e7-9cd0-8ccf1c8371c9 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:32.267693 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae94a2-d6d9-4d31-8f4b-e4daf68b46f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:32.268649 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae94a2-d6d9-4d31-8f4b-e4daf68b46f0 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:32.269107 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae94a2-d6d9-4d31-8f4b-e4daf68b46f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:32.269580 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae94a2-d6d9-4d31-8f4b-e4daf68b46f0 None admin] PATH_INFO: `/v3/groups/f4ca40f956a2466f8aa17cf4de6b03ed` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:32.272171 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ae94a2-d6d9-4d31-8f4b-e4daf68b46f0 None admin] RBAC: Authorizing `identity:get_group(group_id=f4ca40f956a2466f8aa17cf4de6b03ed)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:32.273915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ae94a2-d6d9-4d31-8f4b-e4daf68b46f0 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:32.277159 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 203/394] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Sep 26 10:19:32 2022] GET /identity/v3/groups/f4ca40f956a2466f8aa17cf4de6b03ed => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:32.281830 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae94a2-d6d9-4d31-8f4b-e4daf68b46f0 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:32.383175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79880c64-142e-4c7c-ae7a-dd9b940746bf None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:32.450433 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d831933-901c-4e99-84a1-d7f61469a7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:32.452052 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d831933-901c-4e99-84a1-d7f61469a7c5 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:32.452507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d831933-901c-4e99-84a1-d7f61469a7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:32.452959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d831933-901c-4e99-84a1-d7f61469a7c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:32.465250 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae94a2-d6d9-4d31-8f4b-e4daf68b46f0 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:32.479179 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d831933-901c-4e99-84a1-d7f61469a7c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:32.485054 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-5d831933-901c-4e99-84a1-d7f61469a7c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:19:32.485054 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:32.485054 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:32.485054 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:32.485054 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:32.485054 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:32.485054 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:32.485054 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:32.485054 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:32.485054 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:32.485054 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:32.485054 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:32.485054 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:32.485054 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:32.485054 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:32.485054 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:32.485938 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:32.485938 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:32.485938 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:32.485938 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:32.485938 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:32.485938 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:32.485938 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:32.485938 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:32.485938 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:32.485938 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:32.485938 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:32.485938 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:32.485938 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:32.485938 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:32.485938 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:32.485938 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:32.486774 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:32.486774 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:32.486774 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:32.486774 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:32.486774 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:32.486774 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:32.486774 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:32.486774 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:32.486774 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:32.486774 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:32.486774 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:19:32.486774 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:32.490061 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 192/395] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:19:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:32.495700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d831933-901c-4e99-84a1-d7f61469a7c5 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:32.496735 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:32.524358 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df7747-216b-44ef-ba81-7e2b96b7aa13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:32.525224 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df7747-216b-44ef-ba81-7e2b96b7aa13 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:32.525673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df7747-216b-44ef-ba81-7e2b96b7aa13 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:32.526134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df7747-216b-44ef-ba81-7e2b96b7aa13 None admin] PATH_INFO: `/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:32.527749 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74df7747-216b-44ef-ba81-7e2b96b7aa13 None admin] RBAC: Authorizing `identity:get_project(project_id=f81f2060d8dd4c8a85edacc1547c6b3c)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:32.529939 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74df7747-216b-44ef-ba81-7e2b96b7aa13 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:32.532229 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 204/396] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:19:32 2022] GET /identity/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:32.537847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df7747-216b-44ef-ba81-7e2b96b7aa13 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:32.593120 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:32.665881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df7747-216b-44ef-ba81-7e2b96b7aa13 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:32.679428 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d831933-901c-4e99-84a1-d7f61469a7c5 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:32.734580 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd04260e-2a36-43d9-bfd1-815819ecad06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:32.735244 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04260e-2a36-43d9-bfd1-815819ecad06 None admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:32.735525 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04260e-2a36-43d9-bfd1-815819ecad06 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:32.735798 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04260e-2a36-43d9-bfd1-815819ecad06 None admin] PATH_INFO: `/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c/groups/f4ca40f956a2466f8aa17cf4de6b03ed/roles/d1109db57b004dd9a4f84a634ff76205` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:32.737854 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c8498c-66c1-4dcd-8c55-446bf224bb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:32.738095 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd04260e-2a36-43d9-bfd1-815819ecad06 None admin] RBAC: Authorizing `identity:create_grant(project_id=f81f2060d8dd4c8a85edacc1547c6b3c, group_id=f4ca40f956a2466f8aa17cf4de6b03ed, role_id=d1109db57b004dd9a4f84a634ff76205)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:32.738498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8498c-66c1-4dcd-8c55-446bf224bb09 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:32.738772 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8498c-66c1-4dcd-8c55-446bf224bb09 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:32.739042 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8498c-66c1-4dcd-8c55-446bf224bb09 None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:32.739505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd04260e-2a36-43d9-bfd1-815819ecad06 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:32.739699 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c8498c-66c1-4dcd-8c55-446bf224bb09 None admin] RBAC: Adding query filter params (name=member) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:32.739996 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c8498c-66c1-4dcd-8c55-446bf224bb09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:32.741341 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c8498c-66c1-4dcd-8c55-446bf224bb09 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:32.753454 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 193/397] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 10:19:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:32.754736 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c7dff-3582-467f-98cf-2f4ade7b3014 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:32.755020 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c7dff-3582-467f-98cf-2f4ade7b3014 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:32.755290 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c7dff-3582-467f-98cf-2f4ade7b3014 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:32.759641 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 205/399] 10.222.0.49 () {58 vars in 1442 bytes} [Mon Sep 26 10:19:32 2022] PUT /identity/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c/groups/f4ca40f956a2466f8aa17cf4de6b03ed/roles/d1109db57b004dd9a4f84a634ff76205 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:19:32.760519 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd04260e-2a36-43d9-bfd1-815819ecad06 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:32.760639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 194/399] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:32.761743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f96d42-308a-4194-80fb-01d2676662ce None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:32.761987 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f96d42-308a-4194-80fb-01d2676662ce None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:32.762286 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f96d42-308a-4194-80fb-01d2676662ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:32.799339 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-61f96d42-308a-4194-80fb-01d2676662ce None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:32.800528 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-61f96d42-308a-4194-80fb-01d2676662ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:32.825065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:32.890802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd04260e-2a36-43d9-bfd1-815819ecad06 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:32.905534 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 195/400] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:32.907158 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88802d-55fa-4a32-83ed-ad14799fcc76 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:32.907604 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88802d-55fa-4a32-83ed-ad14799fcc76 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:32.908022 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88802d-55fa-4a32-83ed-ad14799fcc76 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:32.909626 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 196/401] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:32.914962 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132d28b-5bfc-43d2-a464-a43fc8fc9f91 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:32.915506 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132d28b-5bfc-43d2-a464-a43fc8fc9f91 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:32.915999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132d28b-5bfc-43d2-a464-a43fc8fc9f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:32.927775 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68b3da-613a-4ef5-9b3f-7cfb00d4e3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:32.928410 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68b3da-613a-4ef5-9b3f-7cfb00d4e3f9 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:32.928684 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68b3da-613a-4ef5-9b3f-7cfb00d4e3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:32.928954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68b3da-613a-4ef5-9b3f-7cfb00d4e3f9 None admin] PATH_INFO: `/v3/groups/f4ca40f956a2466f8aa17cf4de6b03ed` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:32.930161 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68b3da-613a-4ef5-9b3f-7cfb00d4e3f9 None admin] RBAC: Authorizing `identity:get_group(group_id=f4ca40f956a2466f8aa17cf4de6b03ed)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:32.931653 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68b3da-613a-4ef5-9b3f-7cfb00d4e3f9 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:32.933564 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 206/402] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Sep 26 10:19:32 2022] GET /identity/v3/groups/f4ca40f956a2466f8aa17cf4de6b03ed => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:32.941470 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68b3da-613a-4ef5-9b3f-7cfb00d4e3f9 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:32.942006 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-5132d28b-5bfc-43d2-a464-a43fc8fc9f91 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:32.943439 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-5132d28b-5bfc-43d2-a464-a43fc8fc9f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:33.022871 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 197/403] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 10:19:33.026423 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:33.102077 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68b3da-613a-4ef5-9b3f-7cfb00d4e3f9 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:33.112574 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:33.138920 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf00270b-d813-4020-ad15-9cb0831c6714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:33.139539 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf00270b-d813-4020-ad15-9cb0831c6714 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:33.139821 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf00270b-d813-4020-ad15-9cb0831c6714 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:33.140141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf00270b-d813-4020-ad15-9cb0831c6714 None admin] PATH_INFO: `/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:33.141317 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf00270b-d813-4020-ad15-9cb0831c6714 None admin] RBAC: Authorizing `identity:get_project(project_id=c3a5c629197a4f8ba3a3d1ad27faaa7e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:33.142781 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf00270b-d813-4020-ad15-9cb0831c6714 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:33.146117 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 207/404] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:19:32 2022] GET /identity/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:33.150124 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132d28b-5bfc-43d2-a464-a43fc8fc9f91 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:33.237489 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf00270b-d813-4020-ad15-9cb0831c6714 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:33.254651 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132d28b-5bfc-43d2-a464-a43fc8fc9f91 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:33.290343 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b777f-3f01-45ae-8dfd-002c930ac59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:33.291005 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b777f-3f01-45ae-8dfd-002c930ac59a None admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:33.291282 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b777f-3f01-45ae-8dfd-002c930ac59a None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:33.291553 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b777f-3f01-45ae-8dfd-002c930ac59a None admin] PATH_INFO: `/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e/groups/f4ca40f956a2466f8aa17cf4de6b03ed/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:33.293436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b777f-3f01-45ae-8dfd-002c930ac59a None admin] RBAC: Authorizing `identity:create_grant(project_id=c3a5c629197a4f8ba3a3d1ad27faaa7e, group_id=f4ca40f956a2466f8aa17cf4de6b03ed, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:33.294929 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b777f-3f01-45ae-8dfd-002c930ac59a None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:33.309441 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 198/405] 10.222.0.49 () {58 vars in 1442 bytes} [Mon Sep 26 10:19:33 2022] PUT /identity/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e/groups/f4ca40f956a2466f8aa17cf4de6b03ed/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:19:33.333745 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b777f-3f01-45ae-8dfd-002c930ac59a None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:33.380520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:33.420397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf00270b-d813-4020-ad15-9cb0831c6714 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:33.450311 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b777f-3f01-45ae-8dfd-002c930ac59a None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:33.491306 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ffa6f-d2e8-4bf1-9205-f35608078e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:33.491543 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2ffb4-fe81-42a3-bee8-6b282e9c911a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:33.491958 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ffa6f-d2e8-4bf1-9205-f35608078e76 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:33.492064 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2ffb4-fe81-42a3-bee8-6b282e9c911a None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:33.492230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2ffb4-fe81-42a3-bee8-6b282e9c911a None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:33.492319 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ffa6f-d2e8-4bf1-9205-f35608078e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:33.492492 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ffa6f-d2e8-4bf1-9205-f35608078e76 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:33.492593 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2ffb4-fe81-42a3-bee8-6b282e9c911a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:33.504476 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ffa6f-d2e8-4bf1-9205-f35608078e76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:33.509807 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-139ffa6f-d2e8-4bf1-9205-f35608078e76 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 10:19:33.509807 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:33.509807 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:33.509807 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:33.509807 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:33.509807 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:33.509807 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:33.509807 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:33.509807 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:33.509807 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:33.509807 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:33.509807 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:33.509807 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:33.509807 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:33.509807 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:33.509807 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:33.510743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:33.510743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:33.510743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:33.510743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:33.510743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:33.510743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:33.510743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:33.510743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:33.510743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:33.510743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:33.510743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:33.510743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:33.510743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:33.510743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:33.510743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:33.510743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:33.511556 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:33.511556 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:33.511556 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:33.511556 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:33.511556 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:33.511556 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:33.511556 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:33.511556 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:33.511556 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:33.511556 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:33.511556 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 10:19:33.511556 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:33.511556 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 199/406] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 10:19:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:33.515934 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2ffb4-fe81-42a3-bee8-6b282e9c911a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:33.521404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-0fb2ffb4-fe81-42a3-bee8-6b282e9c911a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:19:33.521404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:33.521404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:33.521404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:33.521404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:33.521404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:33.521404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:33.521404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:33.521404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:33.521404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:33.521404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:33.521404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:33.521404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:33.521404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:33.521404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:33.521404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:33.523550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:33.523550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:33.523550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:33.523550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:33.523550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:33.523550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:33.523550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:33.523550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:33.523550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:33.523550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:33.523550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:33.523550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:33.523550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:33.523550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:33.523550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:33.523550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:33.525479 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:33.525479 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:33.525479 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:33.525479 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:33.525479 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:33.525479 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:33.525479 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:33.525479 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:33.525479 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:33.525479 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:33.525479 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:19:33.525479 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:33.525479 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 208/407] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:19:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:19:33.527106 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ffa6f-d2e8-4bf1-9205-f35608078e76 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:33.531267 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2ffb4-fe81-42a3-bee8-6b282e9c911a None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:33.595218 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:33.633798 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:33.641336 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ffa6f-d2e8-4bf1-9205-f35608078e76 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:33.664928 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2ffb4-fe81-42a3-bee8-6b282e9c911a None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:33.678363 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3dd062-aa92-440b-914a-54e4c067dcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:33.678976 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3dd062-aa92-440b-914a-54e4c067dcb3 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:33.679253 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3dd062-aa92-440b-914a-54e4c067dcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:33.679522 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3dd062-aa92-440b-914a-54e4c067dcb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:33.680186 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3dd062-aa92-440b-914a-54e4c067dcb3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:33.680480 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3dd062-aa92-440b-914a-54e4c067dcb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:33.681846 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3dd062-aa92-440b-914a-54e4c067dcb3 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:33.691832 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 200/408] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:19:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:33.697277 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3dd062-aa92-440b-914a-54e4c067dcb3 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:33.718405 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acfd2bc-c704-48fb-9bdd-ce1aafff5feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:33.719024 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acfd2bc-c704-48fb-9bdd-ce1aafff5feb None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:33.719304 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acfd2bc-c704-48fb-9bdd-ce1aafff5feb None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:33.719569 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acfd2bc-c704-48fb-9bdd-ce1aafff5feb None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:33.720235 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acfd2bc-c704-48fb-9bdd-ce1aafff5feb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:33.720552 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acfd2bc-c704-48fb-9bdd-ce1aafff5feb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:33.722015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acfd2bc-c704-48fb-9bdd-ce1aafff5feb None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:33.741045 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 209/409] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:19:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:33.742081 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acfd2bc-c704-48fb-9bdd-ce1aafff5feb None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:33.863120 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3dd062-aa92-440b-914a-54e4c067dcb3 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:33.880583 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acfd2bc-c704-48fb-9bdd-ce1aafff5feb None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:33.888896 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:33.917986 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecffa7a-1126-44b4-b126-5739f49bbe40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:33.918655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecffa7a-1126-44b4-b126-5739f49bbe40 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:33.918932 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecffa7a-1126-44b4-b126-5739f49bbe40 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:33.919730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecffa7a-1126-44b4-b126-5739f49bbe40 None admin] PATH_INFO: `/v3/groups/6960f263897a4e93ac28513cac7ff71e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:33.920969 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecffa7a-1126-44b4-b126-5739f49bbe40 None admin] RBAC: Authorizing `identity:get_group(group_id=6960f263897a4e93ac28513cac7ff71e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:33.930426 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecffa7a-1126-44b4-b126-5739f49bbe40 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:33.930426 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 210/410] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Sep 26 10:19:33 2022] GET /identity/v3/groups/6960f263897a4e93ac28513cac7ff71e => generated 229 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:33.930426 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecffa7a-1126-44b4-b126-5739f49bbe40 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:33.948391 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49967be0-b81e-4396-8bb8-c69a0724af2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:33.949018 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49967be0-b81e-4396-8bb8-c69a0724af2a None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:33.949351 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49967be0-b81e-4396-8bb8-c69a0724af2a None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:33.949620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49967be0-b81e-4396-8bb8-c69a0724af2a None admin] PATH_INFO: `/v3/groups/f4ca40f956a2466f8aa17cf4de6b03ed` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:33.951104 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49967be0-b81e-4396-8bb8-c69a0724af2a None admin] RBAC: Authorizing `identity:get_group(group_id=f4ca40f956a2466f8aa17cf4de6b03ed)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:33.952544 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49967be0-b81e-4396-8bb8-c69a0724af2a None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:33.954119 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 201/411] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Sep 26 10:19:33 2022] GET /identity/v3/groups/f4ca40f956a2466f8aa17cf4de6b03ed => generated 226 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:33.960151 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49967be0-b81e-4396-8bb8-c69a0724af2a None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:34.014741 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:34.093196 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecffa7a-1126-44b4-b126-5739f49bbe40 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:34.129571 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49967be0-b81e-4396-8bb8-c69a0724af2a None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:34.130264 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf07ad0-c278-4e42-b0b6-9f2b7732236e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:34.130264 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf07ad0-c278-4e42-b0b6-9f2b7732236e None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:34.130523 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf07ad0-c278-4e42-b0b6-9f2b7732236e None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:34.130792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf07ad0-c278-4e42-b0b6-9f2b7732236e None admin] PATH_INFO: `/v3/projects/9cef67c15d1f4b9695af7c22c68b5822` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:34.131921 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf07ad0-c278-4e42-b0b6-9f2b7732236e None admin] RBAC: Authorizing `identity:get_project(project_id=9cef67c15d1f4b9695af7c22c68b5822)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:34.134142 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf07ad0-c278-4e42-b0b6-9f2b7732236e None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:34.135498 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 211/412] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:19:33 2022] GET /identity/v3/projects/9cef67c15d1f4b9695af7c22c68b5822 => generated 344 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:34.142350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf07ad0-c278-4e42-b0b6-9f2b7732236e None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:34.179643 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789a326-9560-4f8d-9976-9b6bce61a37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:34.180277 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789a326-9560-4f8d-9976-9b6bce61a37d None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:34.180554 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789a326-9560-4f8d-9976-9b6bce61a37d None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:34.180825 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789a326-9560-4f8d-9976-9b6bce61a37d None admin] PATH_INFO: `/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:34.181944 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9789a326-9560-4f8d-9976-9b6bce61a37d None admin] RBAC: Authorizing `identity:get_project(project_id=c3a5c629197a4f8ba3a3d1ad27faaa7e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:34.183400 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9789a326-9560-4f8d-9976-9b6bce61a37d None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:34.185533 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 202/413] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:19:33 2022] GET /identity/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:34.193311 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789a326-9560-4f8d-9976-9b6bce61a37d None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:34.216901 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:34.313829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf07ad0-c278-4e42-b0b6-9f2b7732236e None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:34.370313 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:34.379339 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69996cd-bbf0-4b61-b783-652f09d47555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:34.379585 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789a326-9560-4f8d-9976-9b6bce61a37d None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:34.380384 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69996cd-bbf0-4b61-b783-652f09d47555 None admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:34.380812 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69996cd-bbf0-4b61-b783-652f09d47555 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:34.381379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69996cd-bbf0-4b61-b783-652f09d47555 None admin] PATH_INFO: `/v3/projects/9cef67c15d1f4b9695af7c22c68b5822/groups/6960f263897a4e93ac28513cac7ff71e/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:34.384322 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69996cd-bbf0-4b61-b783-652f09d47555 None admin] RBAC: Authorizing `identity:create_grant(project_id=9cef67c15d1f4b9695af7c22c68b5822, group_id=6960f263897a4e93ac28513cac7ff71e, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:34.385899 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69996cd-bbf0-4b61-b783-652f09d47555 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:34.403157 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 212/414] 10.222.0.49 () {58 vars in 1442 bytes} [Mon Sep 26 10:19:34 2022] PUT /identity/v3/projects/9cef67c15d1f4b9695af7c22c68b5822/groups/6960f263897a4e93ac28513cac7ff71e/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:19:34.404882 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3792c-332f-4a6c-a202-189ce281d85b None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:34.405324 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3792c-332f-4a6c-a202-189ce281d85b None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:34.405845 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3792c-332f-4a6c-a202-189ce281d85b None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:34.407346 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 213/415] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:34.415938 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afb48d-b427-4360-b92e-e17cc699e466 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:34.416662 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afb48d-b427-4360-b92e-e17cc699e466 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:34.417097 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afb48d-b427-4360-b92e-e17cc699e466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:34.439691 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de834b-8eef-4dba-b32d-a2265dee8b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:34.440371 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de834b-8eef-4dba-b32d-a2265dee8b4f None admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:34.440647 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de834b-8eef-4dba-b32d-a2265dee8b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:34.440899 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-b3afb48d-b427-4360-b92e-e17cc699e466 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:34.441026 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de834b-8eef-4dba-b32d-a2265dee8b4f None admin] PATH_INFO: `/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e/groups/f4ca40f956a2466f8aa17cf4de6b03ed/roles/d1109db57b004dd9a4f84a634ff76205` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:34.442532 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-b3afb48d-b427-4360-b92e-e17cc699e466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:34.445748 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de834b-8eef-4dba-b32d-a2265dee8b4f None admin] RBAC: Authorizing `identity:create_grant(project_id=c3a5c629197a4f8ba3a3d1ad27faaa7e, group_id=f4ca40f956a2466f8aa17cf4de6b03ed, role_id=d1109db57b004dd9a4f84a634ff76205)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:34.447402 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de834b-8eef-4dba-b32d-a2265dee8b4f None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:34.476253 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 203/416] 10.222.0.49 () {58 vars in 1442 bytes} [Mon Sep 26 10:19:34 2022] PUT /identity/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e/groups/f4ca40f956a2466f8aa17cf4de6b03ed/roles/d1109db57b004dd9a4f84a634ff76205 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:19:34.528077 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 214/417] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:34.910471 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de834b-8eef-4dba-b32d-a2265dee8b4f None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:35.031883 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de834b-8eef-4dba-b32d-a2265dee8b4f None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:35.077127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a8b72f-0673-4722-a406-b2a4ea440f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:35.077696 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8b72f-0673-4722-a406-b2a4ea440f70 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:35.077980 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8b72f-0673-4722-a406-b2a4ea440f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:35.078276 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8b72f-0673-4722-a406-b2a4ea440f70 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:35.094924 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a8b72f-0673-4722-a406-b2a4ea440f70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:35.100384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-42a8b72f-0673-4722-a406-b2a4ea440f70 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:19:35.100384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:35.100384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:35.100384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:35.100384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:35.100384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:35.100384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:35.100384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:35.100384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:35.100384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:35.100384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:35.100384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:35.100384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:35.100384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:35.100384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:35.100384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:35.101244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:35.101244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:35.101244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:35.101244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:35.101244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:35.101244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:35.101244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:35.101244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:35.101244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:35.101244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:35.101244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:35.101244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:35.101244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:35.101244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:35.101244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:35.101244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:35.102125 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:35.102125 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:35.102125 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:35.102125 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:35.102125 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:35.102125 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:35.102125 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:35.102125 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:35.102125 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:35.102125 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:35.102125 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:19:35.102125 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:35.102125 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 204/418] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:19:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:35.109536 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afb48d-b427-4360-b92e-e17cc699e466 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:35.171550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:35.255827 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afb48d-b427-4360-b92e-e17cc699e466 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:35.307790 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76edeec7-4f0b-42ee-acfd-6ee62a106248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:35.308406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edeec7-4f0b-42ee-acfd-6ee62a106248 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:35.308688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edeec7-4f0b-42ee-acfd-6ee62a106248 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:35.308965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edeec7-4f0b-42ee-acfd-6ee62a106248 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:35.309630 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76edeec7-4f0b-42ee-acfd-6ee62a106248 None admin] RBAC: Adding query filter params (name=member) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:35.309965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76edeec7-4f0b-42ee-acfd-6ee62a106248 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:35.311377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76edeec7-4f0b-42ee-acfd-6ee62a106248 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:35.325988 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 215/419] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 10:19:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:35.332584 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a8b72f-0673-4722-a406-b2a4ea440f70 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:35.355158 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:35.467989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a8b72f-0673-4722-a406-b2a4ea440f70 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:35.474596 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368cdde-28aa-43dc-8534-303c2ac2c8fd None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:35.474881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368cdde-28aa-43dc-8534-303c2ac2c8fd None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:35.475160 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368cdde-28aa-43dc-8534-303c2ac2c8fd None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:35.476510 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 216/420] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:19:35.482881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2778d-7dcb-41cb-b368-9f9b0fe9ef97 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:35.483157 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2778d-7dcb-41cb-b368-9f9b0fe9ef97 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:35.483424 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2778d-7dcb-41cb-b368-9f9b0fe9ef97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:35.515785 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-e8a2778d-7dcb-41cb-b368-9f9b0fe9ef97 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:35.517021 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-e8a2778d-7dcb-41cb-b368-9f9b0fe9ef97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:35.535355 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ee328-468b-41d4-9733-f424801641b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:35.535978 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ee328-468b-41d4-9733-f424801641b3 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:35.536257 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ee328-468b-41d4-9733-f424801641b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:35.536528 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ee328-468b-41d4-9733-f424801641b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:35.643111 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 217/421] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:35.670379 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ee328-468b-41d4-9733-f424801641b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:35.671850 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ee328-468b-41d4-9733-f424801641b3 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:35.687834 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:35.781850 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 205/422] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:19:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 10:19:35.787662 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2778d-7dcb-41cb-b368-9f9b0fe9ef97 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:35.827876 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:35.938109 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2778d-7dcb-41cb-b368-9f9b0fe9ef97 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:35.938298 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3cf77-db7b-41a4-81c4-29d6d326c46c None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:35.938563 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3cf77-db7b-41a4-81c4-29d6d326c46c None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:35.938810 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3cf77-db7b-41a4-81c4-29d6d326c46c None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:35.940387 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 206/423] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:35.945427 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b3d01-a8d5-490d-81bc-4124d05c7b7e None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:35.945713 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b3d01-a8d5-490d-81bc-4124d05c7b7e None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:35.946130 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b3d01-a8d5-490d-81bc-4124d05c7b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:35.968810 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-956b3d01-a8d5-490d-81bc-4124d05c7b7e None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:35.970589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-956b3d01-a8d5-490d-81bc-4124d05c7b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:35.974597 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53d2cb-3116-426f-928c-1c9dd809c544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:35.975470 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53d2cb-3116-426f-928c-1c9dd809c544 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:35.975924 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53d2cb-3116-426f-928c-1c9dd809c544 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:35.976375 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53d2cb-3116-426f-928c-1c9dd809c544 None admin] PATH_INFO: `/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:35.977668 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a53d2cb-3116-426f-928c-1c9dd809c544 None admin] RBAC: Authorizing `identity:get_project(project_id=f81f2060d8dd4c8a85edacc1547c6b3c)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:35.979322 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a53d2cb-3116-426f-928c-1c9dd809c544 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:35.981490 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 218/424] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:19:35 2022] GET /identity/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:35.988905 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53d2cb-3116-426f-928c-1c9dd809c544 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:36.055860 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 207/425] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 10:19:36.058160 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b3d01-a8d5-490d-81bc-4124d05c7b7e None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:36.136949 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:36.147590 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:36.163985 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53d2cb-3116-426f-928c-1c9dd809c544 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:36.202318 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df0b48-ed37-4a2d-a09f-7f2bf49101a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:36.204575 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df0b48-ed37-4a2d-a09f-7f2bf49101a9 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:36.205033 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df0b48-ed37-4a2d-a09f-7f2bf49101a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:36.205729 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df0b48-ed37-4a2d-a09f-7f2bf49101a9 None admin] PATH_INFO: `/v3/groups/f4ca40f956a2466f8aa17cf4de6b03ed` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:36.207092 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df0b48-ed37-4a2d-a09f-7f2bf49101a9 None admin] RBAC: Authorizing `identity:get_group(group_id=f4ca40f956a2466f8aa17cf4de6b03ed)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:36.208837 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df0b48-ed37-4a2d-a09f-7f2bf49101a9 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:36.211081 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 219/426] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Sep 26 10:19:35 2022] GET /identity/v3/groups/f4ca40f956a2466f8aa17cf4de6b03ed => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:36.217699 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df0b48-ed37-4a2d-a09f-7f2bf49101a9 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:36.266866 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b3d01-a8d5-490d-81bc-4124d05c7b7e None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:36.319832 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8df3c-1275-4d9b-aa31-5939763318e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:36.320762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8df3c-1275-4d9b-aa31-5939763318e9 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:36.321213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8df3c-1275-4d9b-aa31-5939763318e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:36.321675 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8df3c-1275-4d9b-aa31-5939763318e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:36.334414 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c8df3c-1275-4d9b-aa31-5939763318e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:36.341136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-54c8df3c-1275-4d9b-aa31-5939763318e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 10:19:36.341136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:36.341136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:36.341136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:36.341136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:36.341136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:36.341136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:36.341136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:36.341136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:36.341136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:36.341136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:36.341136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:36.341136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:36.341136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:36.341136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:36.341136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:36.342003 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:36.342003 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:36.342003 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:36.342003 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:36.342003 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:36.342003 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:36.342003 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:36.342003 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:36.342003 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:36.342003 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:36.342003 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:36.342003 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:36.342003 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:36.342003 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:36.342003 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:36.342003 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:36.342849 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:36.342849 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:36.342849 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:36.342849 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:36.342849 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:36.342849 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:36.342849 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:36.342849 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:36.342849 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:36.342849 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:36.342849 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 10:19:36.342849 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:36.345448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 208/427] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 10:19:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:36.350940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8df3c-1275-4d9b-aa31-5939763318e9 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:36.407313 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df0b48-ed37-4a2d-a09f-7f2bf49101a9 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:36.449942 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd24387-7a6b-4a28-b2dc-d7296d64d3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:36.450891 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd24387-7a6b-4a28-b2dc-d7296d64d3c1 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:36.451367 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd24387-7a6b-4a28-b2dc-d7296d64d3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:36.451815 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd24387-7a6b-4a28-b2dc-d7296d64d3c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:36.452779 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd24387-7a6b-4a28-b2dc-d7296d64d3c1 None admin] RBAC: Adding query filter params (group.id=f4ca40f956a2466f8aa17cf4de6b03ed, role.id=dc7cfc688d354df4bc870fe8621c48bd, scope.project.id=f81f2060d8dd4c8a85edacc1547c6b3c) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:36.453269 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd24387-7a6b-4a28-b2dc-d7296d64d3c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:36.454908 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd24387-7a6b-4a28-b2dc-d7296d64d3c1 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:36.461938 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 220/428] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Sep 26 10:19:36 2022] GET /identity/v3/role_assignments?group.id=f4ca40f956a2466f8aa17cf4de6b03ed&scope.project.id=f81f2060d8dd4c8a85edacc1547c6b3c&role.id=dc7cfc688d354df4bc870fe8621c48bd => generated 611 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:36.463031 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd24387-7a6b-4a28-b2dc-d7296d64d3c1 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:36.485666 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:36.495372 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8df3c-1275-4d9b-aa31-5939763318e9 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:36.499511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:36.535506 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213e929-4453-4589-9f01-8e10427e117c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:36.536511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213e929-4453-4589-9f01-8e10427e117c None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:36.536986 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213e929-4453-4589-9f01-8e10427e117c None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:36.537473 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213e929-4453-4589-9f01-8e10427e117c None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:36.538378 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0213e929-4453-4589-9f01-8e10427e117c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:36.538875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0213e929-4453-4589-9f01-8e10427e117c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:36.540608 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0213e929-4453-4589-9f01-8e10427e117c None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:36.550568 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 209/429] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:19:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:36.555666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213e929-4453-4589-9f01-8e10427e117c None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:36.600461 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd24387-7a6b-4a28-b2dc-d7296d64d3c1 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:36.662445 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c18c31-1d9e-463e-beeb-ad45d8dbe8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:36.663509 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c18c31-1d9e-463e-beeb-ad45d8dbe8d5 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:36.664008 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c18c31-1d9e-463e-beeb-ad45d8dbe8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:36.664784 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c18c31-1d9e-463e-beeb-ad45d8dbe8d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:36.679360 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213e929-4453-4589-9f01-8e10427e117c None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:36.698766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c18c31-1d9e-463e-beeb-ad45d8dbe8d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:36.705046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-83c18c31-1d9e-463e-beeb-ad45d8dbe8d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:19:36.705046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:36.705046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:36.705046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:36.705046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:36.705046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:36.705046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:36.705046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:36.705046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:36.705046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:36.705046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:36.705046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:36.705046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:36.705046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:36.705046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:36.705046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:36.706294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:36.706294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:36.706294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:36.706294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:36.706294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:36.706294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:36.706294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:36.706294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:36.706294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:36.706294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:36.706294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:36.706294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:36.706294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:36.706294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:36.706294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:36.706294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:36.707520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:36.707520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:36.707520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:36.707520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:36.707520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:36.707520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:36.707520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:36.707520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:36.707520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:36.707520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:36.707520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 10:19:36.707520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:36.707520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 221/430] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:19:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:36.715030 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c18c31-1d9e-463e-beeb-ad45d8dbe8d5 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:36.733603 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec11fa-f753-4ce4-a5b9-486b77624854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:36.734263 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec11fa-f753-4ce4-a5b9-486b77624854 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:36.734541 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec11fa-f753-4ce4-a5b9-486b77624854 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:36.734811 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec11fa-f753-4ce4-a5b9-486b77624854 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:36.789519 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:36.850165 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ec11fa-f753-4ce4-a5b9-486b77624854 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:36.851626 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ec11fa-f753-4ce4-a5b9-486b77624854 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:36.858716 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:36.889058 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c18c31-1d9e-463e-beeb-ad45d8dbe8d5 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:36.928358 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e151e3-b1dc-456b-95ea-b6c1617049ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:36.928567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e151e3-b1dc-456b-95ea-b6c1617049ff None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:36.928795 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e151e3-b1dc-456b-95ea-b6c1617049ff None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:36.929067 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e151e3-b1dc-456b-95ea-b6c1617049ff None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:36.929730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e151e3-b1dc-456b-95ea-b6c1617049ff None admin] RBAC: Adding query filter params (name=member) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:36.930051 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e151e3-b1dc-456b-95ea-b6c1617049ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:36.932668 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e151e3-b1dc-456b-95ea-b6c1617049ff None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:36.942979 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 222/431] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 10:19:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:36.949470 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e151e3-b1dc-456b-95ea-b6c1617049ff None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:36.996768 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 210/432] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:19:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 10:19:37.002162 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec11fa-f753-4ce4-a5b9-486b77624854 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:37.088545 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e151e3-b1dc-456b-95ea-b6c1617049ff None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:37.097353 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:37.130240 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19d8b0d-5b48-4682-a590-6a41d1fb0f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:37.131027 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d8b0d-5b48-4682-a590-6a41d1fb0f63 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:37.131639 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d8b0d-5b48-4682-a590-6a41d1fb0f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:37.131921 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d8b0d-5b48-4682-a590-6a41d1fb0f63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:37.137269 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec11fa-f753-4ce4-a5b9-486b77624854 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:37.145007 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:37.177010 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d75602-8824-49b0-ba98-c329a19c3e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:37.177647 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d75602-8824-49b0-ba98-c329a19c3e4a None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:37.177919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d75602-8824-49b0-ba98-c329a19c3e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:37.178597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d75602-8824-49b0-ba98-c329a19c3e4a None admin] PATH_INFO: `/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:37.179347 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d75602-8824-49b0-ba98-c329a19c3e4a None admin] RBAC: Authorizing `identity:get_project(project_id=f81f2060d8dd4c8a85edacc1547c6b3c)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:37.181276 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d75602-8824-49b0-ba98-c329a19c3e4a None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:37.182806 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 211/433] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:19:37 2022] GET /identity/v3/projects/f81f2060d8dd4c8a85edacc1547c6b3c => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:37.189241 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d75602-8824-49b0-ba98-c329a19c3e4a None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:37.222930 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19d8b0d-5b48-4682-a590-6a41d1fb0f63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:37.224954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19d8b0d-5b48-4682-a590-6a41d1fb0f63 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:37.320783 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d75602-8824-49b0-ba98-c329a19c3e4a None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:37.365201 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 223/434] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:19:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 10:19:37.366553 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe9538-377d-42ce-abe5-d427a901baed None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:37.366832 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe9538-377d-42ce-abe5-d427a901baed None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:37.367102 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe9538-377d-42ce-abe5-d427a901baed None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:37.368446 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 224/435] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:37.368978 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b4fc7a-941f-439a-8f3d-25b851e2bd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:37.370291 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4fc7a-941f-439a-8f3d-25b851e2bd3c None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:37.370789 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4fc7a-941f-439a-8f3d-25b851e2bd3c None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:37.371267 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4fc7a-941f-439a-8f3d-25b851e2bd3c None admin] PATH_INFO: `/v3/groups/f4ca40f956a2466f8aa17cf4de6b03ed` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:37.373323 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b4fc7a-941f-439a-8f3d-25b851e2bd3c None admin] RBAC: Authorizing `identity:get_group(group_id=f4ca40f956a2466f8aa17cf4de6b03ed)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:37.374793 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe9538-377d-42ce-abe5-d427a901baed None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:37.375493 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b4fc7a-941f-439a-8f3d-25b851e2bd3c None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:37.382293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 212/436] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Sep 26 10:19:37 2022] GET /identity/v3/groups/f4ca40f956a2466f8aa17cf4de6b03ed => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:37.382293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd96bce-c461-48ff-ba3f-4b41de4ae148 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:37.382293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd96bce-c461-48ff-ba3f-4b41de4ae148 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:37.382293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd96bce-c461-48ff-ba3f-4b41de4ae148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:37.387595 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:37.403419 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-efd96bce-c461-48ff-ba3f-4b41de4ae148 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:37.405098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-efd96bce-c461-48ff-ba3f-4b41de4ae148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:37.429000 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:37.483112 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 213/437] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:37.484239 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd96bce-c461-48ff-ba3f-4b41de4ae148 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:37.501377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe9538-377d-42ce-abe5-d427a901baed None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:37.538184 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06bc806-2c49-4d0d-96d4-cf8b332825f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:37.538811 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06bc806-2c49-4d0d-96d4-cf8b332825f7 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:37.539100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06bc806-2c49-4d0d-96d4-cf8b332825f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:37.539391 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06bc806-2c49-4d0d-96d4-cf8b332825f7 None admin] PATH_INFO: `/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:37.540488 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06bc806-2c49-4d0d-96d4-cf8b332825f7 None admin] RBAC: Authorizing `identity:get_project(project_id=c3a5c629197a4f8ba3a3d1ad27faaa7e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:37.541943 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06bc806-2c49-4d0d-96d4-cf8b332825f7 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:37.543826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 225/438] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:19:37 2022] GET /identity/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:37.547569 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77cdba-2798-42e6-9911-e99e2ccba50c None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:37.548502 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77cdba-2798-42e6-9911-e99e2ccba50c None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:37.549132 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77cdba-2798-42e6-9911-e99e2ccba50c None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:37.550944 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 226/439] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:37.552610 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77cdba-2798-42e6-9911-e99e2ccba50c None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:37.590932 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd96bce-c461-48ff-ba3f-4b41de4ae148 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:37.623915 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:37.630665 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab9e6c-09b9-4006-9c30-858dbb7f3721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:37.631267 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab9e6c-09b9-4006-9c30-858dbb7f3721 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:37.631592 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab9e6c-09b9-4006-9c30-858dbb7f3721 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:37.631859 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab9e6c-09b9-4006-9c30-858dbb7f3721 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:37.632632 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab9e6c-09b9-4006-9c30-858dbb7f3721 None admin] RBAC: Adding query filter params (group.id=f4ca40f956a2466f8aa17cf4de6b03ed, role.id=d1109db57b004dd9a4f84a634ff76205, scope.project.id=f81f2060d8dd4c8a85edacc1547c6b3c) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:37.632937 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab9e6c-09b9-4006-9c30-858dbb7f3721 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:37.634533 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab9e6c-09b9-4006-9c30-858dbb7f3721 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:37.650295 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 214/440] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Sep 26 10:19:37 2022] GET /identity/v3/role_assignments?group.id=f4ca40f956a2466f8aa17cf4de6b03ed&scope.project.id=f81f2060d8dd4c8a85edacc1547c6b3c&role.id=d1109db57b004dd9a4f84a634ff76205 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:37.650295 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d09e1-4888-41c1-be3e-b948aba9dc46 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:37.650295 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d09e1-4888-41c1-be3e-b948aba9dc46 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:37.650295 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d09e1-4888-41c1-be3e-b948aba9dc46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:37.665949 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-315d09e1-4888-41c1-be3e-b948aba9dc46 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:37.667499 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-315d09e1-4888-41c1-be3e-b948aba9dc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:37.701731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77cdba-2798-42e6-9911-e99e2ccba50c None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:37.715320 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:37.745647 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb8173-8c8a-4060-81ef-f0a877e4d70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:37.746344 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb8173-8c8a-4060-81ef-f0a877e4d70e None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:37.746669 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb8173-8c8a-4060-81ef-f0a877e4d70e None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:37.746938 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb8173-8c8a-4060-81ef-f0a877e4d70e None admin] PATH_INFO: `/v3/groups/f4ca40f956a2466f8aa17cf4de6b03ed` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:37.748783 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cb8173-8c8a-4060-81ef-f0a877e4d70e None admin] RBAC: Authorizing `identity:get_group(group_id=f4ca40f956a2466f8aa17cf4de6b03ed)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:37.751019 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cb8173-8c8a-4060-81ef-f0a877e4d70e None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:37.753297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 227/441] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Sep 26 10:19:37 2022] GET /identity/v3/groups/f4ca40f956a2466f8aa17cf4de6b03ed => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:37.759203 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb8173-8c8a-4060-81ef-f0a877e4d70e None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:37.759949 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 215/442] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:37.769546 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d09e1-4888-41c1-be3e-b948aba9dc46 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:37.885388 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb8173-8c8a-4060-81ef-f0a877e4d70e None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:37.892055 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d09e1-4888-41c1-be3e-b948aba9dc46 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:37.927186 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:37.932267 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1cd4c-c726-4b82-a302-f69fbe04f241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:37.932880 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1cd4c-c726-4b82-a302-f69fbe04f241 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:37.933251 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1cd4c-c726-4b82-a302-f69fbe04f241 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:37.933529 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1cd4c-c726-4b82-a302-f69fbe04f241 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:37.934379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d1cd4c-c726-4b82-a302-f69fbe04f241 None admin] RBAC: Adding query filter params (group.id=f4ca40f956a2466f8aa17cf4de6b03ed, role.id=dc7cfc688d354df4bc870fe8621c48bd, scope.project.id=c3a5c629197a4f8ba3a3d1ad27faaa7e) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:37.934482 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d1cd4c-c726-4b82-a302-f69fbe04f241 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:37.935967 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d1cd4c-c726-4b82-a302-f69fbe04f241 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:37.938098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd687b-445c-4527-975d-7b88b1a93ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:37.939431 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd687b-445c-4527-975d-7b88b1a93ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:37.939431 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd687b-445c-4527-975d-7b88b1a93ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:37.939431 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd687b-445c-4527-975d-7b88b1a93ee0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:37.943162 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 228/443] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Sep 26 10:19:37 2022] GET /identity/v3/role_assignments?group.id=f4ca40f956a2466f8aa17cf4de6b03ed&scope.project.id=c3a5c629197a4f8ba3a3d1ad27faaa7e&role.id=dc7cfc688d354df4bc870fe8621c48bd => generated 611 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:37.952284 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd687b-445c-4527-975d-7b88b1a93ee0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:37.957580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-83bd687b-445c-4527-975d-7b88b1a93ee0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:19:37.957580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:37.957580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:37.957580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:37.957580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:37.957580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:37.957580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:37.957580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:37.957580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:37.957580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:37.957580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:37.957580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:37.957580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:37.957580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:37.957580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:37.957580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:37.958597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:37.958597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:37.958597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:37.958597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:37.958597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:37.958597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:37.958597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:37.958597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:37.958597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:37.958597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:37.958597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:37.958597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:37.958597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:37.958597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:37.958597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:37.958597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:37.959387 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:37.959387 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:37.959387 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:37.959387 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:37.959387 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:37.959387 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:37.959387 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:37.959387 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:37.959387 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:37.959387 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:37.959387 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:19:37.959387 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:37.959387 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 216/444] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:19:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:37.967417 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1cd4c-c726-4b82-a302-f69fbe04f241 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:38.020035 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:38.067487 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd687b-445c-4527-975d-7b88b1a93ee0 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:38.097168 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1cd4c-c726-4b82-a302-f69fbe04f241 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:38.148773 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3649b-5e3c-4dcc-9f1a-add0a1b824cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:38.148773 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3649b-5e3c-4dcc-9f1a-add0a1b824cf None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:38.148773 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3649b-5e3c-4dcc-9f1a-add0a1b824cf None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:38.148773 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3649b-5e3c-4dcc-9f1a-add0a1b824cf None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:38.149158 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3649b-5e3c-4dcc-9f1a-add0a1b824cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:38.149158 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3649b-5e3c-4dcc-9f1a-add0a1b824cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:38.149742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3649b-5e3c-4dcc-9f1a-add0a1b824cf None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:38.160271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 229/445] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:19:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:38.166253 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3649b-5e3c-4dcc-9f1a-add0a1b824cf None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:38.210089 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd687b-445c-4527-975d-7b88b1a93ee0 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:38.256690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab0824-cdd7-411e-ad4e-2d95a09b6101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:38.257315 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab0824-cdd7-411e-ad4e-2d95a09b6101 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:38.260796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab0824-cdd7-411e-ad4e-2d95a09b6101 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:38.260796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab0824-cdd7-411e-ad4e-2d95a09b6101 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:38.268721 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab0824-cdd7-411e-ad4e-2d95a09b6101 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:38.273523 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-6fab0824-cdd7-411e-ad4e-2d95a09b6101 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 10:19:38.273523 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:38.273523 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:38.273523 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:38.273523 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:38.273523 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:38.273523 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:38.273523 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:38.273523 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:38.273523 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:38.273523 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:38.273523 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:38.273523 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:38.273523 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:38.273523 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:38.273523 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:38.274629 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:38.274629 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:38.274629 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:38.274629 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:38.274629 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:38.274629 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:38.274629 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:38.274629 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:38.274629 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:38.274629 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:38.274629 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:38.274629 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:38.274629 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:38.274629 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:38.274629 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:38.274629 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:38.275433 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:38.275433 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:38.275433 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:38.275433 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:38.275433 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:38.275433 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:38.275433 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:38.275433 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:38.275433 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:38.275433 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:38.275433 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 10:19:38.275433 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:38.275433 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 217/446] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 10:19:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:19:38.280554 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab0824-cdd7-411e-ad4e-2d95a09b6101 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:38.286860 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3649b-5e3c-4dcc-9f1a-add0a1b824cf None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:38.288974 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:38.333322 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0e5e0-98a2-4d3e-95d3-3c8207fe88ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:38.334343 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0e5e0-98a2-4d3e-95d3-3c8207fe88ea None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:38.335337 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0e5e0-98a2-4d3e-95d3-3c8207fe88ea None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:38.335337 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0e5e0-98a2-4d3e-95d3-3c8207fe88ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:38.336909 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:38.399511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab0824-cdd7-411e-ad4e-2d95a09b6101 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:38.409113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd0e5e0-98a2-4d3e-95d3-3c8207fe88ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:38.410594 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd0e5e0-98a2-4d3e-95d3-3c8207fe88ea None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:38.432824 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5bc76a-d19e-4af1-9b39-a5233d68f04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:38.433421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5bc76a-d19e-4af1-9b39-a5233d68f04a None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:38.433692 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5bc76a-d19e-4af1-9b39-a5233d68f04a None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:38.433956 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5bc76a-d19e-4af1-9b39-a5233d68f04a None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:38.434635 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5bc76a-d19e-4af1-9b39-a5233d68f04a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:38.434990 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5bc76a-d19e-4af1-9b39-a5233d68f04a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:38.436850 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5bc76a-d19e-4af1-9b39-a5233d68f04a None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:38.444936 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 218/447] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:19:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:38.451566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5bc76a-d19e-4af1-9b39-a5233d68f04a None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:38.508090 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 230/448] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:19:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 10:19:38.514158 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0e5e0-98a2-4d3e-95d3-3c8207fe88ea None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:38.516002 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:38.550032 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5bc76a-d19e-4af1-9b39-a5233d68f04a None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:38.558795 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:38.584625 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003964b5-f830-4fc1-ace2-e1ac8e0dde1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:38.585228 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003964b5-f830-4fc1-ace2-e1ac8e0dde1e None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:38.585509 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003964b5-f830-4fc1-ace2-e1ac8e0dde1e None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:38.585783 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003964b5-f830-4fc1-ace2-e1ac8e0dde1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:38.614181 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0e5e0-98a2-4d3e-95d3-3c8207fe88ea None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:38.647797 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e593094-6b6f-403e-8f08-50d03ca881c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:38.648418 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e593094-6b6f-403e-8f08-50d03ca881c2 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:38.648696 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e593094-6b6f-403e-8f08-50d03ca881c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:38.648968 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e593094-6b6f-403e-8f08-50d03ca881c2 None admin] PATH_INFO: `/v3/projects/9cef67c15d1f4b9695af7c22c68b5822` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:38.650173 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e593094-6b6f-403e-8f08-50d03ca881c2 None admin] RBAC: Authorizing `identity:get_project(project_id=9cef67c15d1f4b9695af7c22c68b5822)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:38.651562 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e593094-6b6f-403e-8f08-50d03ca881c2 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:38.651882 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003964b5-f830-4fc1-ace2-e1ac8e0dde1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:38.653180 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 231/449] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:19:38 2022] GET /identity/v3/projects/9cef67c15d1f4b9695af7c22c68b5822 => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:38.653310 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003964b5-f830-4fc1-ace2-e1ac8e0dde1e None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:38.658777 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e593094-6b6f-403e-8f08-50d03ca881c2 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:38.728544 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:38.752989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 219/450] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:19:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 10:19:38.758378 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003964b5-f830-4fc1-ace2-e1ac8e0dde1e None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:38.760496 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e593094-6b6f-403e-8f08-50d03ca881c2 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:38.768175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:38.793304 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f10a74-a1f6-4c12-a8a5-ac11a0d87c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:38.793940 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f10a74-a1f6-4c12-a8a5-ac11a0d87c97 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:38.794286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f10a74-a1f6-4c12-a8a5-ac11a0d87c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:38.794559 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f10a74-a1f6-4c12-a8a5-ac11a0d87c97 None admin] PATH_INFO: `/v3/groups/6960f263897a4e93ac28513cac7ff71e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:38.795692 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f10a74-a1f6-4c12-a8a5-ac11a0d87c97 None admin] RBAC: Authorizing `identity:get_group(group_id=6960f263897a4e93ac28513cac7ff71e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:38.797130 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f10a74-a1f6-4c12-a8a5-ac11a0d87c97 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:38.798727 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 232/451] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Sep 26 10:19:38 2022] GET /identity/v3/groups/6960f263897a4e93ac28513cac7ff71e => generated 229 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:38.804512 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f10a74-a1f6-4c12-a8a5-ac11a0d87c97 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:38.858827 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003964b5-f830-4fc1-ace2-e1ac8e0dde1e None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:38.891614 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad479b8-caa5-45b0-bff5-c1c88e4260e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:38.892228 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad479b8-caa5-45b0-bff5-c1c88e4260e1 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:38.892497 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad479b8-caa5-45b0-bff5-c1c88e4260e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:38.892764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad479b8-caa5-45b0-bff5-c1c88e4260e1 None admin] PATH_INFO: `/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:38.893870 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad479b8-caa5-45b0-bff5-c1c88e4260e1 None admin] RBAC: Authorizing `identity:get_project(project_id=c3a5c629197a4f8ba3a3d1ad27faaa7e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:38.895344 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad479b8-caa5-45b0-bff5-c1c88e4260e1 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:38.896944 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 220/452] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:19:38 2022] GET /identity/v3/projects/c3a5c629197a4f8ba3a3d1ad27faaa7e => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:38.902060 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f10a74-a1f6-4c12-a8a5-ac11a0d87c97 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:38.902438 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad479b8-caa5-45b0-bff5-c1c88e4260e1 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:38.937262 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655650a2-ac2f-4d9e-8978-6d88bae3a1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:38.938036 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655650a2-ac2f-4d9e-8978-6d88bae3a1a6 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:38.939065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655650a2-ac2f-4d9e-8978-6d88bae3a1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:38.939344 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655650a2-ac2f-4d9e-8978-6d88bae3a1a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:38.940066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655650a2-ac2f-4d9e-8978-6d88bae3a1a6 None admin] RBAC: Adding query filter params (group.id=6960f263897a4e93ac28513cac7ff71e, role.id=532b37d9076c45618cf352aafb85f35e, scope.project.id=9cef67c15d1f4b9695af7c22c68b5822) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:38.940370 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655650a2-ac2f-4d9e-8978-6d88bae3a1a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:38.941785 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655650a2-ac2f-4d9e-8978-6d88bae3a1a6 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:38.947565 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 233/453] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Sep 26 10:19:38 2022] GET /identity/v3/role_assignments?group.id=6960f263897a4e93ac28513cac7ff71e&scope.project.id=9cef67c15d1f4b9695af7c22c68b5822&role.id=532b37d9076c45618cf352aafb85f35e => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:38.982283 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:39.006832 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad479b8-caa5-45b0-bff5-c1c88e4260e1 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:39.042658 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53a0b9-2267-47b6-a4e2-f9e61ea35052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:39.043291 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53a0b9-2267-47b6-a4e2-f9e61ea35052 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:39.043568 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53a0b9-2267-47b6-a4e2-f9e61ea35052 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:39.043838 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53a0b9-2267-47b6-a4e2-f9e61ea35052 None admin] PATH_INFO: `/v3/groups/f4ca40f956a2466f8aa17cf4de6b03ed` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:39.045544 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d53a0b9-2267-47b6-a4e2-f9e61ea35052 None admin] RBAC: Authorizing `identity:get_group(group_id=f4ca40f956a2466f8aa17cf4de6b03ed)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:39.047047 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d53a0b9-2267-47b6-a4e2-f9e61ea35052 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:39.048664 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 221/454] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Sep 26 10:19:38 2022] GET /identity/v3/groups/f4ca40f956a2466f8aa17cf4de6b03ed => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:39.054577 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655650a2-ac2f-4d9e-8978-6d88bae3a1a6 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:39.057476 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:39.162009 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655650a2-ac2f-4d9e-8978-6d88bae3a1a6 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:39.195748 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43571dff-b790-4fcd-a4de-67b8813541fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:39.196345 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43571dff-b790-4fcd-a4de-67b8813541fa None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:39.196625 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43571dff-b790-4fcd-a4de-67b8813541fa None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:39.196898 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43571dff-b790-4fcd-a4de-67b8813541fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:39.197612 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43571dff-b790-4fcd-a4de-67b8813541fa None admin] RBAC: Adding query filter params (group.id=f4ca40f956a2466f8aa17cf4de6b03ed, role.id=d1109db57b004dd9a4f84a634ff76205, scope.project.id=c3a5c629197a4f8ba3a3d1ad27faaa7e) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:39.197917 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43571dff-b790-4fcd-a4de-67b8813541fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:39.199401 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43571dff-b790-4fcd-a4de-67b8813541fa None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:39.204992 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 234/455] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Sep 26 10:19:39 2022] GET /identity/v3/role_assignments?group.id=f4ca40f956a2466f8aa17cf4de6b03ed&scope.project.id=c3a5c629197a4f8ba3a3d1ad27faaa7e&role.id=d1109db57b004dd9a4f84a634ff76205 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:41.841707 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb6b27-818f-418f-903d-761272aea26e None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:41.842267 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb6b27-818f-418f-903d-761272aea26e None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:41.842267 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb6b27-818f-418f-903d-761272aea26e None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:41.843608 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 222/456] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:41.859689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8904e4f-8c88-4c1f-a00f-c6b2b96d3329 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:41.860157 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8904e4f-8c88-4c1f-a00f-c6b2b96d3329 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:41.860231 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8904e4f-8c88-4c1f-a00f-c6b2b96d3329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:41.887051 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-a8904e4f-8c88-4c1f-a00f-c6b2b96d3329 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:41.888836 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-a8904e4f-8c88-4c1f-a00f-c6b2b96d3329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:41.967595 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:41.976488 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 235/457] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:42.213670 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a04fc-1e8a-43c9-9e72-e92433a1ad60 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:42.214194 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a04fc-1e8a-43c9-9e72-e92433a1ad60 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:42.214637 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a04fc-1e8a-43c9-9e72-e92433a1ad60 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:42.216540 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 223/458] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:42.222903 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d41de-d618-4180-a924-0dd0b38483ce None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:42.223478 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d41de-d618-4180-a924-0dd0b38483ce None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:42.223949 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d41de-d618-4180-a924-0dd0b38483ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:42.246144 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-dd1d41de-d618-4180-a924-0dd0b38483ce None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:42.247745 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-dd1d41de-d618-4180-a924-0dd0b38483ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:42.317911 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a061afc-d623-466a-bd26-2aad01a93c9c None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:42.318496 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a061afc-d623-466a-bd26-2aad01a93c9c None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:42.318989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a061afc-d623-466a-bd26-2aad01a93c9c None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:42.321980 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 224/459] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:42.327011 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19054fa0-433c-4924-89ff-01d5ad394110 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:42.327529 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19054fa0-433c-4924-89ff-01d5ad394110 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:42.328020 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19054fa0-433c-4924-89ff-01d5ad394110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:42.354122 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 236/460] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:42.359971 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-19054fa0-433c-4924-89ff-01d5ad394110 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:42.361387 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-19054fa0-433c-4924-89ff-01d5ad394110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:42.419834 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d41de-d618-4180-a924-0dd0b38483ce None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:42.487121 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 225/461] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 10:19:42.589360 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d41de-d618-4180-a924-0dd0b38483ce None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:42.620902 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:42.631712 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff854deb-1ecd-4149-a41f-99c63835b78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:42.632632 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff854deb-1ecd-4149-a41f-99c63835b78f None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:42.633109 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff854deb-1ecd-4149-a41f-99c63835b78f None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:42.633574 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff854deb-1ecd-4149-a41f-99c63835b78f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:42.644734 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff854deb-1ecd-4149-a41f-99c63835b78f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:42.646457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff854deb-1ecd-4149-a41f-99c63835b78f None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:42.662381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-ff854deb-1ecd-4149-a41f-99c63835b78f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:42.662381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:42.662381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:42.662381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:42.662381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:42.662381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:42.662381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:42.662381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:42.662381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:42.662381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:42.662381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:42.662381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:42.662381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:19:42.662381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:19:42.662381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:19:42.662381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:19:42.663271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:42.663271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:42.663271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:42.663271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:42.663271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:42.663271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:42.663271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:42.663271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:42.663271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:42.663271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:42.663271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:42.663271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:42.663271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:42.663271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:42.663271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:42.663271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:42.663271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:19:42.664194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:19:42.664194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:42.664194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:42.670115 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 237/462] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:42.672424 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19054fa0-433c-4924-89ff-01d5ad394110 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:42.689946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff854deb-1ecd-4149-a41f-99c63835b78f None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:42.753817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:42.780789 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19054fa0-433c-4924-89ff-01d5ad394110 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:42.834717 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5c47b-c1cf-4e63-a0df-877c16e81a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:42.835328 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5c47b-c1cf-4e63-a0df-877c16e81a3c None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:42.835613 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5c47b-c1cf-4e63-a0df-877c16e81a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:42.835975 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5c47b-c1cf-4e63-a0df-877c16e81a3c None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:42.836635 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df5c47b-c1cf-4e63-a0df-877c16e81a3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:42.836937 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df5c47b-c1cf-4e63-a0df-877c16e81a3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:42.838465 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df5c47b-c1cf-4e63-a0df-877c16e81a3c None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:42.855585 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 226/463] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:19:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:42.857338 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5c47b-c1cf-4e63-a0df-877c16e81a3c None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:42.858954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff854deb-1ecd-4149-a41f-99c63835b78f None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:42.904538 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc52fb-48d7-49d7-8555-3db65115ca94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:42.905251 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc52fb-48d7-49d7-8555-3db65115ca94 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:42.905744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc52fb-48d7-49d7-8555-3db65115ca94 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:42.906379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc52fb-48d7-49d7-8555-3db65115ca94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:42.920712 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc52fb-48d7-49d7-8555-3db65115ca94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:42.922766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc52fb-48d7-49d7-8555-3db65115ca94 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:42.940393 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-dcbc52fb-48d7-49d7-8555-3db65115ca94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:42.940393 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:42.940393 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:42.940393 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:42.940393 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:42.940393 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:42.940393 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:42.940393 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:42.940393 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:42.940393 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:42.940393 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:42.940393 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:42.940393 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:19:42.940393 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:19:42.940393 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:19:42.940393 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:19:42.941368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:42.941368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:42.941368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:42.941368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:42.941368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:42.941368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:42.941368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:42.941368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:42.941368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:42.941368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:42.941368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:42.941368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:42.941368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:42.941368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:42.941368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:42.941368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:42.941368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:19:42.942239 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:19:42.942239 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:42.942239 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:42.944520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 238/464] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:19:42.945567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc52fb-48d7-49d7-8555-3db65115ca94 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:42.981273 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:42.988086 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5c47b-c1cf-4e63-a0df-877c16e81a3c None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:43.032228 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622770bf-324c-4209-ba57-ee0450438d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:43.032841 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622770bf-324c-4209-ba57-ee0450438d5d None admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:43.033121 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622770bf-324c-4209-ba57-ee0450438d5d None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:43.033395 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622770bf-324c-4209-ba57-ee0450438d5d None admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:43.034128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622770bf-324c-4209-ba57-ee0450438d5d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:43.036009 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622770bf-324c-4209-ba57-ee0450438d5d None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:43.046637 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:43.063050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 227/465] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 10:19:42 2022] POST /identity/v3/users => generated 246 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:19:43.063559 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc52fb-48d7-49d7-8555-3db65115ca94 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:43.064214 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622770bf-324c-4209-ba57-ee0450438d5d None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:43.109391 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8857a1-09e2-41a9-ba73-2bb9ecbaddb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:43.110436 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8857a1-09e2-41a9-ba73-2bb9ecbaddb4 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:43.110975 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8857a1-09e2-41a9-ba73-2bb9ecbaddb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:43.111451 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8857a1-09e2-41a9-ba73-2bb9ecbaddb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:43.126001 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8857a1-09e2-41a9-ba73-2bb9ecbaddb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:43.128055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8857a1-09e2-41a9-ba73-2bb9ecbaddb4 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:43.149480 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-7a8857a1-09e2-41a9-ba73-2bb9ecbaddb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:43.149480 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:43.149480 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:43.149480 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:43.149480 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:43.149480 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:43.149480 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:43.149480 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:43.149480 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:43.149480 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:43.149480 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:43.149480 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:43.149480 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:19:43.149480 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:19:43.149480 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:19:43.149480 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:19:43.150523 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:43.150523 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:43.150523 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:43.150523 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:43.150523 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:43.150523 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:43.150523 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:43.150523 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:43.150523 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:43.150523 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:43.150523 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:43.150523 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:43.150523 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:43.150523 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:43.150523 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:43.150523 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:43.150523 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:19:43.151387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:19:43.151387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:43.151387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:43.154298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 239/466] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:43.155866 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067d451-d68f-4b5d-bae6-3be5f21bb0fa None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:43.156384 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067d451-d68f-4b5d-bae6-3be5f21bb0fa None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:43.156854 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067d451-d68f-4b5d-bae6-3be5f21bb0fa None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:43.158904 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 240/467] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:43.159884 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067d451-d68f-4b5d-bae6-3be5f21bb0fa None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:43.215330 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622770bf-324c-4209-ba57-ee0450438d5d None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:43.259430 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:43.264380 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34597ba1-a0aa-46a7-8fe1-4fedd1dc6794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:43.264995 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34597ba1-a0aa-46a7-8fe1-4fedd1dc6794 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:43.265274 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34597ba1-a0aa-46a7-8fe1-4fedd1dc6794 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:43.265546 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34597ba1-a0aa-46a7-8fe1-4fedd1dc6794 None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:43.266869 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34597ba1-a0aa-46a7-8fe1-4fedd1dc6794 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:43.266869 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34597ba1-a0aa-46a7-8fe1-4fedd1dc6794 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:43.268399 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34597ba1-a0aa-46a7-8fe1-4fedd1dc6794 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:43.283254 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 228/468] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:19:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:43.284221 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fed161-b096-4c3f-9bae-5e8881233d2e None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:43.284508 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fed161-b096-4c3f-9bae-5e8881233d2e None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:43.284783 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fed161-b096-4c3f-9bae-5e8881233d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:43.289873 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067d451-d68f-4b5d-bae6-3be5f21bb0fa None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:43.303903 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-f2fed161-b096-4c3f-9bae-5e8881233d2e None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:43.305132 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-f2fed161-b096-4c3f-9bae-5e8881233d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:43.341839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6cf1e-8306-4dce-a750-6e32b6cd02fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:43.342743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6cf1e-8306-4dce-a750-6e32b6cd02fc None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:43.343168 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6cf1e-8306-4dce-a750-6e32b6cd02fc None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:43.343577 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6cf1e-8306-4dce-a750-6e32b6cd02fc None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:43.344433 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c6cf1e-8306-4dce-a750-6e32b6cd02fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:43.344940 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c6cf1e-8306-4dce-a750-6e32b6cd02fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:43.346822 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c6cf1e-8306-4dce-a750-6e32b6cd02fc None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:43.362159 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 241/469] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:19:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:43.363226 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6cf1e-8306-4dce-a750-6e32b6cd02fc None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:43.366425 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:43.393658 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 229/470] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:43.399597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fed161-b096-4c3f-9bae-5e8881233d2e None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:43.519772 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6cf1e-8306-4dce-a750-6e32b6cd02fc None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:43.536390 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fed161-b096-4c3f-9bae-5e8881233d2e None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:43.555575 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f91cd-6555-480f-9c32-465771813f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:43.556186 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f91cd-6555-480f-9c32-465771813f19 None admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:43.556471 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f91cd-6555-480f-9c32-465771813f19 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:43.556746 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f91cd-6555-480f-9c32-465771813f19 None admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:43.557397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f91cd-6555-480f-9c32-465771813f19 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:43.558926 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f91cd-6555-480f-9c32-465771813f19 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:43.578483 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:43.583625 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1fcad-78fc-434a-ac4f-542451fc5ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:43.584448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1fcad-78fc-434a-ac4f-542451fc5ce8 None admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:43.584985 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 242/471] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 10:19:43 2022] POST /identity/v3/users => generated 248 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:19:43.585557 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1fcad-78fc-434a-ac4f-542451fc5ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:43.586149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1fcad-78fc-434a-ac4f-542451fc5ce8 None admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:43.586746 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5dcf5-97d2-4d9c-90a7-27d380eb8017 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:43.587247 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5dcf5-97d2-4d9c-90a7-27d380eb8017 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:43.587549 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1fcad-78fc-434a-ac4f-542451fc5ce8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:43.587657 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5dcf5-97d2-4d9c-90a7-27d380eb8017 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:43.590570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1fcad-78fc-434a-ac4f-542451fc5ce8 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:43.594373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 243/472] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:43 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:43.595928 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5c5b2-3b79-4418-808a-eeb691629b36 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:43.596308 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5c5b2-3b79-4418-808a-eeb691629b36 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:43.596676 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5c5b2-3b79-4418-808a-eeb691629b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:43.619583 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-cde5c5b2-3b79-4418-808a-eeb691629b36 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:43.620981 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-cde5c5b2-3b79-4418-808a-eeb691629b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:43.621843 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 230/473] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 10:19:43 2022] POST /identity/v3/users => generated 248 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:19:43.713254 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 244/474] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 10:19:43.757573 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1fcad-78fc-434a-ac4f-542451fc5ce8 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:43.822870 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:43.905041 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1fcad-78fc-434a-ac4f-542451fc5ce8 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:43.942132 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287fb4c-f4d2-4376-8d7f-2447271341dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:43.942774 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287fb4c-f4d2-4376-8d7f-2447271341dd None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:43.943055 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287fb4c-f4d2-4376-8d7f-2447271341dd None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:43.943331 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287fb4c-f4d2-4376-8d7f-2447271341dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:43.954871 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1287fb4c-f4d2-4376-8d7f-2447271341dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:43.956606 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1287fb4c-f4d2-4376-8d7f-2447271341dd None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:43.972813 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-1287fb4c-f4d2-4376-8d7f-2447271341dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:43.972813 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:43.972813 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:43.972813 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:43.972813 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:43.972813 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:43.972813 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:43.972813 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:43.972813 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:43.972813 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:43.972813 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:43.972813 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:43.972813 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:19:43.972813 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:19:43.972813 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:19:43.972813 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:19:43.973821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:43.973821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:43.973821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:43.973821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:43.973821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:43.973821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:43.973821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:43.973821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:43.973821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:43.973821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:43.973821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:43.973821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:43.973821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:43.973821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:43.973821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:43.973821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:43.973821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:19:43.974747 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:19:43.974747 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:43.974747 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:43.974747 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 231/475] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:43.979856 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5c5b2-3b79-4418-808a-eeb691629b36 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:44.001248 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287fb4c-f4d2-4376-8d7f-2447271341dd None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:44.093485 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5c5b2-3b79-4418-808a-eeb691629b36 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:44.103498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:44.118659 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287fb4c-f4d2-4376-8d7f-2447271341dd None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:44.128811 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30745932-9722-4f93-b8d6-b79ef24de6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:44.129531 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30745932-9722-4f93-b8d6-b79ef24de6a0 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:44.129711 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30745932-9722-4f93-b8d6-b79ef24de6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:44.129948 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30745932-9722-4f93-b8d6-b79ef24de6a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:44.130626 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30745932-9722-4f93-b8d6-b79ef24de6a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:44.130924 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30745932-9722-4f93-b8d6-b79ef24de6a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:44.132361 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30745932-9722-4f93-b8d6-b79ef24de6a0 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:44.141149 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:44.144533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 245/476] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:19:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:44.149911 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30745932-9722-4f93-b8d6-b79ef24de6a0 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:44.165555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9e7295-2d6a-4f75-85d6-d5bcc8c593e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:44.166200 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e7295-2d6a-4f75-85d6-d5bcc8c593e6 None admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:44.166480 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e7295-2d6a-4f75-85d6-d5bcc8c593e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:44.166749 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e7295-2d6a-4f75-85d6-d5bcc8c593e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:44.167428 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9e7295-2d6a-4f75-85d6-d5bcc8c593e6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:44.168821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9e7295-2d6a-4f75-85d6-d5bcc8c593e6 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:44.186300 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.sql.core [None req-4d9e7295-2d6a-4f75-85d6-d5bcc8c593e6 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 26 10:19:44.186300 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 26 10:19:44.186300 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [parameters: {'id': '7c97e15c6d0f4b379641dbe362e0c93e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 26 10:19:44.186300 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81421) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 26 10:19:44.218576 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-4d9e7295-2d6a-4f75-85d6-d5bcc8c593e6 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 26 10:19:44.218576 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:44.218576 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 26 10:19:44.218576 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 26 10:19:44.218576 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 26 10:19:44.218576 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 26 10:19:44.218576 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 26 10:19:44.218576 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application result = self._query(query) Sep 26 10:19:44.218576 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 26 10:19:44.218576 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application conn.query(q) Sep 26 10:19:44.218576 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 26 10:19:44.218576 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 26 10:19:44.218576 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 26 10:19:44.218576 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application result.read() Sep 26 10:19:44.218576 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 26 10:19:44.219507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 26 10:19:44.219507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 26 10:19:44.219507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 26 10:19:44.219507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 26 10:19:44.219507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 26 10:19:44.219507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 26 10:19:44.219507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 26 10:19:44.219507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 26 10:19:44.219507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:44.219507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 26 10:19:44.219507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:44.219507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:44.219507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 26 10:19:44.219507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 26 10:19:44.219507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 26 10:19:44.219507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return ref.to_dict() Sep 26 10:19:44.219507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 26 10:19:44.219507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application next(self.gen) Sep 26 10:19:44.220463 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 26 10:19:44.220463 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application yield resource Sep 26 10:19:44.220463 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 26 10:19:44.220463 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application next(self.gen) Sep 26 10:19:44.220463 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 26 10:19:44.220463 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application self.session.rollback() Sep 26 10:19:44.220463 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 26 10:19:44.220463 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application self.force_reraise() Sep 26 10:19:44.220463 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 26 10:19:44.220463 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise self.value Sep 26 10:19:44.220463 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 26 10:19:44.220463 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 26 10:19:44.220463 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 26 10:19:44.220463 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application session.commit() Sep 26 10:19:44.220463 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 26 10:19:44.220463 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 26 10:19:44.220463 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 26 10:19:44.221325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application self._prepare_impl() Sep 26 10:19:44.221325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 26 10:19:44.221325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application self.session.flush() Sep 26 10:19:44.221325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 26 10:19:44.221325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application self._flush(objects) Sep 26 10:19:44.221325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 26 10:19:44.221325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 26 10:19:44.221325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 26 10:19:44.221325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application compat.raise_( Sep 26 10:19:44.221325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 26 10:19:44.221325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception Sep 26 10:19:44.221325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 26 10:19:44.221325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application flush_context.execute() Sep 26 10:19:44.221325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 26 10:19:44.221325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rec.execute(self) Sep 26 10:19:44.221325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 26 10:19:44.221325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 26 10:19:44.222989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 26 10:19:44.222989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 26 10:19:44.222989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 26 10:19:44.222989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 26 10:19:44.222989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 26 10:19:44.222989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 26 10:19:44.222989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 26 10:19:44.222989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 26 10:19:44.222989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 26 10:19:44.222989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 26 10:19:44.222989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 26 10:19:44.222989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 26 10:19:44.222989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 26 10:19:44.222989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 26 10:19:44.222989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 26 10:19:44.222989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception Sep 26 10:19:44.223834 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 26 10:19:44.223834 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 26 10:19:44.223834 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 26 10:19:44.223834 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 26 10:19:44.223834 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 26 10:19:44.223834 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application result = self._query(query) Sep 26 10:19:44.223834 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 26 10:19:44.223834 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application conn.query(q) Sep 26 10:19:44.223834 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 26 10:19:44.223834 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 26 10:19:44.223834 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 26 10:19:44.223834 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application result.read() Sep 26 10:19:44.223834 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 26 10:19:44.223834 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 26 10:19:44.223834 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 26 10:19:44.223834 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 26 10:19:44.224636 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 26 10:19:44.224636 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 26 10:19:44.224636 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 26 10:19:44.224636 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 26 10:19:44.224636 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 26 10:19:44.224636 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 26 10:19:44.224636 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [parameters: {'id': '7c97e15c6d0f4b379641dbe362e0c93e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 26 10:19:44.224636 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 26 10:19:44.224636 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:44.224636 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 10:19:44.224636 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:44.224636 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:44.224636 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:44.224636 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:44.224636 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:44.224636 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:44.226931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:44.226931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:44.226931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:44.226931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:44.226931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:44.226931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:44.226931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 26 10:19:44.226931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 26 10:19:44.226931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:44.226931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:44.226931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 26 10:19:44.226931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 26 10:19:44.226931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 26 10:19:44.226931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 26 10:19:44.226931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 26 10:19:44.226931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:44.228529 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 232/477] 10.222.0.49 () {60 vars in 1131 bytes} [Mon Sep 26 10:19:44 2022] POST /identity/v3/roles => generated 144 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:44.228529 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9e7295-2d6a-4f75-85d6-d5bcc8c593e6 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:44.257941 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30745932-9722-4f93-b8d6-b79ef24de6a0 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:44.295255 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40dae7-9bb3-4f82-9ae7-69425ef478a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:44.295854 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40dae7-9bb3-4f82-9ae7-69425ef478a1 None admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:44.296130 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40dae7-9bb3-4f82-9ae7-69425ef478a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:44.296398 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40dae7-9bb3-4f82-9ae7-69425ef478a1 None admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:44.297052 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e40dae7-9bb3-4f82-9ae7-69425ef478a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:44.298725 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e40dae7-9bb3-4f82-9ae7-69425ef478a1 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:44.334937 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 246/478] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 10:19:44 2022] POST /identity/v3/users => generated 249 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:19:44.412194 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9e7295-2d6a-4f75-85d6-d5bcc8c593e6 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:44.466929 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06810508-432d-4600-8e17-7057a91ebb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:44.467936 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06810508-432d-4600-8e17-7057a91ebb17 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:44.468433 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06810508-432d-4600-8e17-7057a91ebb17 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:44.468893 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06810508-432d-4600-8e17-7057a91ebb17 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:44.490721 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06810508-432d-4600-8e17-7057a91ebb17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:44.496655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-06810508-432d-4600-8e17-7057a91ebb17 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 10:19:44.496655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:44.496655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:44.496655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:44.496655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:44.496655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:44.496655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:44.496655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:44.496655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:44.496655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:44.496655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:44.496655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:44.496655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:44.496655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:44.496655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:44.496655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:44.497621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:44.497621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:44.497621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:44.497621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:44.497621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:44.497621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:44.497621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:44.497621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:44.497621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:44.497621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:44.497621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:44.497621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:44.497621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:44.497621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:44.497621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:44.497621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:44.498457 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:44.498457 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:44.498457 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:44.498457 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:44.498457 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:44.498457 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:44.498457 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:44.498457 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:44.498457 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:44.498457 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:44.498457 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 10:19:44.498457 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:44.500954 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 233/479] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 10:19:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:44.506977 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40dae7-9bb3-4f82-9ae7-69425ef478a1 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:44.571998 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:44.631200 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40dae7-9bb3-4f82-9ae7-69425ef478a1 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:44.666296 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c440af9-70e4-4194-b188-0d31a030a951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:44.666824 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c440af9-70e4-4194-b188-0d31a030a951 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:44.667100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c440af9-70e4-4194-b188-0d31a030a951 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:44.667370 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c440af9-70e4-4194-b188-0d31a030a951 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:44.668032 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c440af9-70e4-4194-b188-0d31a030a951 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:44.668331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c440af9-70e4-4194-b188-0d31a030a951 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:44.669731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c440af9-70e4-4194-b188-0d31a030a951 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:44.678580 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 247/480] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:19:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:45.652449 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f03f00-f1f5-4e49-9911-c6d07c377cce None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:45.653279 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f03f00-f1f5-4e49-9911-c6d07c377cce None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:45.654378 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f03f00-f1f5-4e49-9911-c6d07c377cce None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:45.656117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 234/481] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:45.661654 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf2b4e-815d-492c-8a5f-cfa2e3585251 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:45.662104 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf2b4e-815d-492c-8a5f-cfa2e3585251 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:45.662435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf2b4e-815d-492c-8a5f-cfa2e3585251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:45.691384 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-94bf2b4e-815d-492c-8a5f-cfa2e3585251 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:45.693034 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-94bf2b4e-815d-492c-8a5f-cfa2e3585251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:45.770860 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 248/482] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:46.174076 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f03f00-f1f5-4e49-9911-c6d07c377cce None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:46.189751 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:46.310702 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f03f00-f1f5-4e49-9911-c6d07c377cce None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:46.360257 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01066e7c-ae61-410e-b2b2-0f0de0d2b36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:46.361272 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01066e7c-ae61-410e-b2b2-0f0de0d2b36c None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:46.361740 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01066e7c-ae61-410e-b2b2-0f0de0d2b36c None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:46.362237 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01066e7c-ae61-410e-b2b2-0f0de0d2b36c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:46.379346 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01066e7c-ae61-410e-b2b2-0f0de0d2b36c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:46.384451 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-01066e7c-ae61-410e-b2b2-0f0de0d2b36c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:19:46.384451 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:46.384451 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:46.384451 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:46.384451 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:46.384451 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:46.384451 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:46.384451 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:46.384451 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:46.384451 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:46.384451 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:46.384451 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:46.384451 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:46.384451 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:46.384451 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:46.384451 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:46.385296 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:46.385296 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:46.385296 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:46.385296 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:46.385296 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:46.385296 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:46.385296 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:46.385296 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:46.385296 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:46.385296 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:46.385296 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:46.385296 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:46.385296 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:46.385296 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:46.385296 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:46.385296 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:46.386117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:46.386117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:46.386117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:46.386117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:46.386117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:46.386117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:46.386117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:46.386117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:46.386117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:46.386117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:46.386117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:19:46.386117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:46.388281 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 235/483] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 10:19:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:46.393748 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf2b4e-815d-492c-8a5f-cfa2e3585251 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:46.459710 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:46.533544 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf2b4e-815d-492c-8a5f-cfa2e3585251 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:46.537385 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7774294-506c-4eca-b268-84169ee62ba5 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:46.537780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7774294-506c-4eca-b268-84169ee62ba5 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:46.538090 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7774294-506c-4eca-b268-84169ee62ba5 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:46.540794 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 236/484] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:46.545476 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad2ce3-cc05-4994-87e0-9aeea1f6e531 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:46.545759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad2ce3-cc05-4994-87e0-9aeea1f6e531 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:46.546059 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad2ce3-cc05-4994-87e0-9aeea1f6e531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:46.569391 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e72549-a48e-48b6-bfe0-e1c31031ffa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:46.570006 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e72549-a48e-48b6-bfe0-e1c31031ffa3 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:46.570328 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e72549-a48e-48b6-bfe0-e1c31031ffa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:46.570600 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e72549-a48e-48b6-bfe0-e1c31031ffa3 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:46.571266 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e72549-a48e-48b6-bfe0-e1c31031ffa3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:46.571564 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e72549-a48e-48b6-bfe0-e1c31031ffa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:46.572937 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e72549-a48e-48b6-bfe0-e1c31031ffa3 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:46.583269 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 249/485] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 10:19:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:46.588815 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e72549-a48e-48b6-bfe0-e1c31031ffa3 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:46.589852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-7fad2ce3-cc05-4994-87e0-9aeea1f6e531 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:46.591172 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-7fad2ce3-cc05-4994-87e0-9aeea1f6e531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:46.702784 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:46.704862 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e72549-a48e-48b6-bfe0-e1c31031ffa3 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:46.730183 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 237/486] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:46.732625 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df4059-0256-4ef4-8997-7f1b96b1957c None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:46.733105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df4059-0256-4ef4-8997-7f1b96b1957c None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:46.733568 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df4059-0256-4ef4-8997-7f1b96b1957c None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:46.735261 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 238/487] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:46.740018 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852472e-8a9a-45e1-a1fa-1172f4f39c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:46.740797 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852472e-8a9a-45e1-a1fa-1172f4f39c62 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:46.741167 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852472e-8a9a-45e1-a1fa-1172f4f39c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:46.741531 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852472e-8a9a-45e1-a1fa-1172f4f39c62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:46.751893 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af07749-2d4f-413f-82ef-270a079a76fb None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:46.752386 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af07749-2d4f-413f-82ef-270a079a76fb None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:46.752795 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af07749-2d4f-413f-82ef-270a079a76fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:46.753626 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3852472e-8a9a-45e1-a1fa-1172f4f39c62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:46.755208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3852472e-8a9a-45e1-a1fa-1172f4f39c62 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:46.765944 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:46.779677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-3852472e-8a9a-45e1-a1fa-1172f4f39c62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:46.779677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:46.779677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:46.779677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:46.779677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:46.779677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:46.779677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:46.779677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:46.779677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:46.779677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:46.779677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:46.779677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:46.779677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:19:46.779677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:19:46.779677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:19:46.779677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:19:46.780576 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:46.780576 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:46.780576 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:46.780576 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:46.780576 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:46.780576 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:46.780576 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:46.780576 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:46.780576 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:46.780576 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:46.780576 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:46.780576 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:46.780576 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:46.780576 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:46.780576 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:46.780576 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:46.780576 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:19:46.781440 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:19:46.781440 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:46.781440 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:46.782002 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 250/488] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:19:46.786285 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-6af07749-2d4f-413f-82ef-270a079a76fb None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:46.787559 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-6af07749-2d4f-413f-82ef-270a079a76fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:46.790278 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852472e-8a9a-45e1-a1fa-1172f4f39c62 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:46.913087 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 239/489] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:46.949011 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852472e-8a9a-45e1-a1fa-1172f4f39c62 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:46.986623 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1beb3-ed8b-47d2-8231-3fcfb531047f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:46.987143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1beb3-ed8b-47d2-8231-3fcfb531047f None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:46.987416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1beb3-ed8b-47d2-8231-3fcfb531047f None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:46.987685 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1beb3-ed8b-47d2-8231-3fcfb531047f None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:46.988318 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1beb3-ed8b-47d2-8231-3fcfb531047f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:46.988615 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1beb3-ed8b-47d2-8231-3fcfb531047f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:46.989977 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1beb3-ed8b-47d2-8231-3fcfb531047f None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:47.003100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 251/490] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:19:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:47.011350 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af07749-2d4f-413f-82ef-270a079a76fb None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:47.064231 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e105f-acb0-4e07-97c4-5be5237558d4 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:47.064537 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e105f-acb0-4e07-97c4-5be5237558d4 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:47.064771 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e105f-acb0-4e07-97c4-5be5237558d4 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:47.066180 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 252/491] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:19:47.071145 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26274b1-05b7-4423-b906-1a6049659bbf None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:47.071421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26274b1-05b7-4423-b906-1a6049659bbf None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:47.071692 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26274b1-05b7-4423-b906-1a6049659bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:47.093507 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-c26274b1-05b7-4423-b906-1a6049659bbf None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:47.094772 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-c26274b1-05b7-4423-b906-1a6049659bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:47.097617 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:47.137927 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af07749-2d4f-413f-82ef-270a079a76fb None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:47.170055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 253/492] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 10:19:47.171039 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26274b1-05b7-4423-b906-1a6049659bbf None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:47.174932 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:47.180625 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4f95f-1779-47f4-a3a1-426255c3b76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:47.181265 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4f95f-1779-47f4-a3a1-426255c3b76d None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:47.181648 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4f95f-1779-47f4-a3a1-426255c3b76d None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:47.182732 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4f95f-1779-47f4-a3a1-426255c3b76d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:47.190598 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b4f95f-1779-47f4-a3a1-426255c3b76d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:47.192060 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b4f95f-1779-47f4-a3a1-426255c3b76d None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:47.206554 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-49b4f95f-1779-47f4-a3a1-426255c3b76d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 10:19:47.206554 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:47.206554 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:47.206554 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:47.206554 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:47.206554 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:47.206554 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:47.206554 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:47.206554 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:47.206554 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:47.206554 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:47.206554 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:47.206554 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:19:47.206554 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:19:47.206554 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:19:47.206554 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:19:47.207458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:47.207458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:47.207458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:19:47.207458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:19:47.207458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:19:47.207458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:19:47.207458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:47.207458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:47.207458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:47.207458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:47.207458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:47.207458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:47.207458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:47.207458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:47.207458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:47.207458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:47.207458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:47.207458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:47.208412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:47.208412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:47.208412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:19:47.208412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:19:47.208412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:19:47.208412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:19:47.208412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:19:47.208412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:19:47.208412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 10:19:47.208412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:47.208412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 240/493] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Sep 26 10:19:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:47.215644 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4f95f-1779-47f4-a3a1-426255c3b76d None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:47.276964 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26274b1-05b7-4423-b906-1a6049659bbf None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:47.313943 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba073397-1663-4120-8c37-292d1403711a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:47.314604 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba073397-1663-4120-8c37-292d1403711a None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:47.314884 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba073397-1663-4120-8c37-292d1403711a None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:47.315157 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba073397-1663-4120-8c37-292d1403711a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:47.323359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:47.328396 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba073397-1663-4120-8c37-292d1403711a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:47.333517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-ba073397-1663-4120-8c37-292d1403711a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:19:47.333517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:47.333517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:47.333517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:47.333517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:47.333517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:47.333517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:47.333517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:47.333517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:47.333517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:47.333517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:47.333517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:47.333517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:47.333517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:47.333517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:47.333517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:47.335491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:47.335491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:47.335491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:47.335491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:47.335491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:47.335491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:47.335491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:47.335491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:47.335491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:47.335491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:47.335491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:47.335491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:47.335491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:47.335491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:47.335491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:47.335491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:47.337310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:47.337310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:47.337310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:47.337310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:47.337310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:47.337310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:47.337310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:47.337310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:47.337310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:47.337310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:47.337310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:19:47.337310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:47.337310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 254/494] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 10:19:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:47.344968 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba073397-1663-4120-8c37-292d1403711a None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:47.367102 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4f95f-1779-47f4-a3a1-426255c3b76d None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:47.413668 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4e7a2-0835-478a-9f76-1a7dea4a9179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:47.414394 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4e7a2-0835-478a-9f76-1a7dea4a9179 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:47.414937 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4e7a2-0835-478a-9f76-1a7dea4a9179 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:47.415211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4e7a2-0835-478a-9f76-1a7dea4a9179 None admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:47.415945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c4e7a2-0835-478a-9f76-1a7dea4a9179 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:47.416497 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c4e7a2-0835-478a-9f76-1a7dea4a9179 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:47.418823 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c4e7a2-0835-478a-9f76-1a7dea4a9179 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:47.455295 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba073397-1663-4120-8c37-292d1403711a None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:47.461774 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 241/495] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Sep 26 10:19:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:47.462726 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4e7a2-0835-478a-9f76-1a7dea4a9179 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:47.497239 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b3d26-37ac-4ef8-9209-7b0a894eb260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:47.497922 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b3d26-37ac-4ef8-9209-7b0a894eb260 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:47.498240 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b3d26-37ac-4ef8-9209-7b0a894eb260 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:47.498522 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b3d26-37ac-4ef8-9209-7b0a894eb260 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:47.509823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0b3d26-37ac-4ef8-9209-7b0a894eb260 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:47.515327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-7c0b3d26-37ac-4ef8-9209-7b0a894eb260 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:19:47.515327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:47.515327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:47.515327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:47.515327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:47.515327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:47.515327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:47.515327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:47.515327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:47.515327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:47.515327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:47.515327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:47.515327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:47.515327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:47.515327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:47.515327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:47.516217 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:47.516217 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:47.516217 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:47.516217 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:47.516217 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:47.516217 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:47.516217 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:47.516217 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:47.516217 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:47.516217 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:47.516217 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:47.516217 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:47.516217 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:47.516217 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:47.516217 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:47.516217 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:47.516997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:47.516997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:47.516997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:47.516997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:47.516997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:47.516997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:47.516997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:47.516997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:47.516997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:47.516997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:47.516997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:19:47.516997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:47.516997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 255/496] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 10:19:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:47.517671 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b3d26-37ac-4ef8-9209-7b0a894eb260 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:47.529951 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:47.579923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4e7a2-0835-478a-9f76-1a7dea4a9179 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:47.586987 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:47.614677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84cc7a-9fd4-4d81-bbd5-0f3a33b45324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:47.615285 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84cc7a-9fd4-4d81-bbd5-0f3a33b45324 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:47.615566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84cc7a-9fd4-4d81-bbd5-0f3a33b45324 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:47.615841 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84cc7a-9fd4-4d81-bbd5-0f3a33b45324 None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:47.616502 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e84cc7a-9fd4-4d81-bbd5-0f3a33b45324 None admin] RBAC: Adding query filter params (name=service) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:47.616804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e84cc7a-9fd4-4d81-bbd5-0f3a33b45324 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:47.618217 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e84cc7a-9fd4-4d81-bbd5-0f3a33b45324 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:47.631042 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 242/497] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 10:19:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:47.631042 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84cc7a-9fd4-4d81-bbd5-0f3a33b45324 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:47.631265 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b3d26-37ac-4ef8-9209-7b0a894eb260 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:47.663258 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e5ef0f-f8aa-4cf9-9ce9-512848b6aa97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:47.664072 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e5ef0f-f8aa-4cf9-9ce9-512848b6aa97 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:47.664345 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e5ef0f-f8aa-4cf9-9ce9-512848b6aa97 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:47.664610 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e5ef0f-f8aa-4cf9-9ce9-512848b6aa97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:47.729003 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84cc7a-9fd4-4d81-bbd5-0f3a33b45324 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:47.735068 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e5ef0f-f8aa-4cf9-9ce9-512848b6aa97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:47.736538 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e5ef0f-f8aa-4cf9-9ce9-512848b6aa97 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:47.741507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:47.764099 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbdd054-1d71-4d3b-a809-422d4488586f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:47.764727 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdd054-1d71-4d3b-a809-422d4488586f None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:47.765006 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdd054-1d71-4d3b-a809-422d4488586f None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:47.765278 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdd054-1d71-4d3b-a809-422d4488586f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:47.775118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbdd054-1d71-4d3b-a809-422d4488586f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:47.780436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-4fbdd054-1d71-4d3b-a809-422d4488586f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:19:47.780436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:47.780436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:47.780436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:47.780436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:47.780436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:47.780436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:47.780436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:47.780436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:47.780436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:47.780436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:47.780436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:47.780436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:47.780436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:47.780436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:47.780436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:47.781475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:47.781475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:47.781475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:47.781475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:47.781475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:47.781475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:47.781475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:47.781475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:47.781475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:47.781475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:47.781475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:47.781475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:47.781475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:47.781475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:47.781475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:47.781475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:47.782450 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:47.782450 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:47.782450 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:47.782450 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:47.782450 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:47.782450 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:47.782450 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:47.782450 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:47.782450 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:47.782450 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:47.782450 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:19:47.782450 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:47.782450 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 243/498] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 10:19:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:47.782450 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbdd054-1d71-4d3b-a809-422d4488586f None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:47.800587 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:47.844664 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 256/499] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:19:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 10:19:47.846136 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24698171-5520-4eed-a66f-594f438313b5 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:47.846424 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24698171-5520-4eed-a66f-594f438313b5 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:47.846699 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24698171-5520-4eed-a66f-594f438313b5 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:47.847929 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 257/500] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:47.848612 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24698171-5520-4eed-a66f-594f438313b5 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:47.886296 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbdd054-1d71-4d3b-a809-422d4488586f None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:47.919475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a80ed3-4f24-43df-a761-8461b7e1e020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:47.920253 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a80ed3-4f24-43df-a761-8461b7e1e020 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:47.920543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a80ed3-4f24-43df-a761-8461b7e1e020 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:47.920813 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a80ed3-4f24-43df-a761-8461b7e1e020 None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:47.921460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a80ed3-4f24-43df-a761-8461b7e1e020 None admin] RBAC: Adding query filter params (name=service) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:47.921757 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a80ed3-4f24-43df-a761-8461b7e1e020 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:47.923144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a80ed3-4f24-43df-a761-8461b7e1e020 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:47.930999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 244/501] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 10:19:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:47.931794 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a80ed3-4f24-43df-a761-8461b7e1e020 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:47.956457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24698171-5520-4eed-a66f-594f438313b5 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:47.966311 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:47.990101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b280a2-7788-4301-ac48-f797dbbf39f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:47.990719 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b280a2-7788-4301-ac48-f797dbbf39f2 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:47.990995 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b280a2-7788-4301-ac48-f797dbbf39f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:47.991268 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b280a2-7788-4301-ac48-f797dbbf39f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:48.001837 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b280a2-7788-4301-ac48-f797dbbf39f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:48.003309 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b280a2-7788-4301-ac48-f797dbbf39f2 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:48.018108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-18b280a2-7788-4301-ac48-f797dbbf39f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:48.018108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:48.018108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:48.018108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:48.018108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:48.018108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:48.018108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:48.018108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:48.018108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:48.018108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:48.018108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:48.018108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:48.018108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:19:48.018108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:19:48.018108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:19:48.018108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:19:48.019065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:48.019065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:48.019065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:48.019065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:48.019065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:48.019065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:48.019065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:48.019065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:48.019065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:48.019065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:48.019065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:48.019065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:48.019065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:48.019065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:48.019065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:48.019065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:48.019065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:19:48.019906 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:19:48.019906 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:48.019906 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:48.019906 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 258/502] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:48.019906 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b280a2-7788-4301-ac48-f797dbbf39f2 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:48.034510 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:48.036716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a80ed3-4f24-43df-a761-8461b7e1e020 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:48.070919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3bcc6-5c87-47db-9083-d1376a1cb1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:48.071520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3bcc6-5c87-47db-9083-d1376a1cb1b2 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:48.071798 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3bcc6-5c87-47db-9083-d1376a1cb1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:48.072069 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3bcc6-5c87-47db-9083-d1376a1cb1b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:48.072721 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3bcc6-5c87-47db-9083-d1376a1cb1b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:48.073021 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3bcc6-5c87-47db-9083-d1376a1cb1b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:48.074422 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3bcc6-5c87-47db-9083-d1376a1cb1b2 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:48.082585 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 245/503] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 10:19:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:48.083709 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3dd4e-2cea-4955-9d1b-731fece8cee3 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:48.084001 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3dd4e-2cea-4955-9d1b-731fece8cee3 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:48.084280 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3dd4e-2cea-4955-9d1b-731fece8cee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:48.101446 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-59f3dd4e-2cea-4955-9d1b-731fece8cee3 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:48.102699 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-59f3dd4e-2cea-4955-9d1b-731fece8cee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:48.120402 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b280a2-7788-4301-ac48-f797dbbf39f2 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:48.154207 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3da3fd-5539-4595-bdb6-3f21358d56a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:48.154510 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3da3fd-5539-4595-bdb6-3f21358d56a7 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:48.154785 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3da3fd-5539-4595-bdb6-3f21358d56a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:48.155063 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3da3fd-5539-4595-bdb6-3f21358d56a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:48.165099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3da3fd-5539-4595-bdb6-3f21358d56a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:48.166563 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3da3fd-5539-4595-bdb6-3f21358d56a7 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:48.166923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 246/504] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:48.167697 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3dd4e-2cea-4955-9d1b-731fece8cee3 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:48.188505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-3e3da3fd-5539-4595-bdb6-3f21358d56a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:48.188505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:48.188505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:48.188505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:48.188505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:48.188505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:48.188505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:48.188505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:48.188505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:48.188505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:48.188505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:48.188505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:48.188505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:19:48.188505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:19:48.188505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:19:48.188505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:19:48.189432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:48.189432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:48.189432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:48.189432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:48.189432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:48.189432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:48.189432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:48.189432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:48.189432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:48.189432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:48.189432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:48.189432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:48.189432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:48.189432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:48.189432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:48.189432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:48.189432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:19:48.190358 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:19:48.190358 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:48.190358 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:48.190358 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 259/505] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:48.190358 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3da3fd-5539-4595-bdb6-3f21358d56a7 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:48.203745 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:48.256301 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:48.281368 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3dd4e-2cea-4955-9d1b-731fece8cee3 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:48.298352 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3da3fd-5539-4595-bdb6-3f21358d56a7 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:48.316514 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28faff36-0ce7-4eba-8626-c8f957726883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:48.317433 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28faff36-0ce7-4eba-8626-c8f957726883 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:48.317887 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28faff36-0ce7-4eba-8626-c8f957726883 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:48.318375 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28faff36-0ce7-4eba-8626-c8f957726883 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:48.329521 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28faff36-0ce7-4eba-8626-c8f957726883 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:48.331325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28faff36-0ce7-4eba-8626-c8f957726883 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:48.339383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b1db1-6f01-4542-ad32-e09f29644a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:48.339984 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b1db1-6f01-4542-ad32-e09f29644a62 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:48.340260 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b1db1-6f01-4542-ad32-e09f29644a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:48.340529 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b1db1-6f01-4542-ad32-e09f29644a62 None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:48.341188 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b1db1-6f01-4542-ad32-e09f29644a62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:48.341482 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b1db1-6f01-4542-ad32-e09f29644a62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:48.342891 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b1db1-6f01-4542-ad32-e09f29644a62 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:48.346146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-28faff36-0ce7-4eba-8626-c8f957726883 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:48.346146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:48.346146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:48.346146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:48.346146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:48.346146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:48.346146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:48.346146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:48.346146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:48.346146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:48.346146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:48.346146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:48.346146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:19:48.346146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:19:48.346146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:19:48.346146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:19:48.346998 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:48.346998 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:48.346998 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:48.346998 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:48.346998 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:48.346998 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:48.346998 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:48.346998 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:48.346998 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:48.346998 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:48.346998 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:48.346998 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:48.346998 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:48.346998 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:48.346998 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:48.346998 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:48.346998 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:19:48.352692 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:19:48.352692 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:48.352692 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:48.352692 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 247/506] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:48.352692 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28faff36-0ce7-4eba-8626-c8f957726883 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:48.357737 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 260/507] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:19:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:48.362310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b1db1-6f01-4542-ad32-e09f29644a62 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:48.441485 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:48.460879 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b1db1-6f01-4542-ad32-e09f29644a62 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:48.463316 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28faff36-0ce7-4eba-8626-c8f957726883 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:48.484525 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:48.495471 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4d335-1b5b-458e-af2e-bf1b5cf1f13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:48.496070 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4d335-1b5b-458e-af2e-bf1b5cf1f13f None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:48.496298 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845a756-9586-4ed2-b52c-5a844288561e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:48.496416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4d335-1b5b-458e-af2e-bf1b5cf1f13f None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:48.496605 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4d335-1b5b-458e-af2e-bf1b5cf1f13f None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:48.496910 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845a756-9586-4ed2-b52c-5a844288561e None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:48.497183 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845a756-9586-4ed2-b52c-5a844288561e None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:48.497313 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4d335-1b5b-458e-af2e-bf1b5cf1f13f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:48.497470 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845a756-9586-4ed2-b52c-5a844288561e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:48.497588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4d335-1b5b-458e-af2e-bf1b5cf1f13f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:48.498927 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4d335-1b5b-458e-af2e-bf1b5cf1f13f None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:48.507994 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2845a756-9586-4ed2-b52c-5a844288561e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:48.509428 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2845a756-9586-4ed2-b52c-5a844288561e None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:48.510197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 261/508] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:19:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:48.510980 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4d335-1b5b-458e-af2e-bf1b5cf1f13f None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:48.523690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-2845a756-9586-4ed2-b52c-5a844288561e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:48.523690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:48.523690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:48.523690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:48.523690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:48.523690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:48.523690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:48.523690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:48.523690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:48.523690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:48.523690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:48.523690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:48.523690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:19:48.523690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:19:48.523690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:19:48.523690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:19:48.524599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:48.524599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:48.524599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:48.524599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:48.524599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:48.524599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:48.524599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:48.524599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:48.524599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:48.524599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:48.524599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:48.524599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:48.524599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:48.524599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:48.524599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:48.524599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:48.524599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:19:48.525474 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:19:48.525474 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:48.525474 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:48.525474 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 248/509] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:48.525916 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845a756-9586-4ed2-b52c-5a844288561e None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:48.614542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4d335-1b5b-458e-af2e-bf1b5cf1f13f None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:48.623803 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845a756-9586-4ed2-b52c-5a844288561e None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:48.646871 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc260fa-d6e0-4cfe-8637-55a28e25e66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:48.647469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc260fa-d6e0-4cfe-8637-55a28e25e66f None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:48.647747 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc260fa-d6e0-4cfe-8637-55a28e25e66f None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:48.648020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc260fa-d6e0-4cfe-8637-55a28e25e66f None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:48.648659 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc260fa-d6e0-4cfe-8637-55a28e25e66f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:48.648959 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc260fa-d6e0-4cfe-8637-55a28e25e66f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:48.650362 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc260fa-d6e0-4cfe-8637-55a28e25e66f None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:48.651387 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:48.658085 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2bfa7f-eb14-47ae-9436-23e2bc532bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:48.658699 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2bfa7f-eb14-47ae-9436-23e2bc532bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:48.658974 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2bfa7f-eb14-47ae-9436-23e2bc532bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:48.659244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2bfa7f-eb14-47ae-9436-23e2bc532bd6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:48.662446 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 262/510] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:19:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:48.663234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc260fa-d6e0-4cfe-8637-55a28e25e66f None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:48.666622 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2bfa7f-eb14-47ae-9436-23e2bc532bd6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:48.668067 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2bfa7f-eb14-47ae-9436-23e2bc532bd6 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:48.679808 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-1b2bfa7f-eb14-47ae-9436-23e2bc532bd6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 26 10:19:48.679808 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:48.679808 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:48.679808 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:48.679808 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:48.679808 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:48.679808 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:48.679808 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:48.679808 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:48.679808 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:48.679808 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:48.679808 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:48.679808 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:19:48.679808 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:19:48.679808 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:19:48.679808 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:19:48.680785 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:48.680785 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:48.680785 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:19:48.680785 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:19:48.680785 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:19:48.680785 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:19:48.680785 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:48.680785 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:48.680785 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:48.680785 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:48.680785 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:48.680785 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:48.680785 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:48.680785 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:48.680785 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:48.680785 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:48.680785 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:48.680785 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:48.681666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:48.681666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:48.681666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:19:48.681666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:19:48.681666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:19:48.681666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:19:48.681666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:19:48.681666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:19:48.681666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 26 10:19:48.681666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:48.681666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 249/511] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:19:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:48.681666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2bfa7f-eb14-47ae-9436-23e2bc532bd6 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:48.712222 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:48.766709 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc260fa-d6e0-4cfe-8637-55a28e25e66f None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:48.783464 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2bfa7f-eb14-47ae-9436-23e2bc532bd6 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:48.800698 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1708de4-f39a-46bc-9014-80200ef519b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:48.801448 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1708de4-f39a-46bc-9014-80200ef519b4 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:48.801791 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1708de4-f39a-46bc-9014-80200ef519b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:48.802161 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1708de4-f39a-46bc-9014-80200ef519b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:48.812365 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1708de4-f39a-46bc-9014-80200ef519b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:48.813806 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1708de4-f39a-46bc-9014-80200ef519b4 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:48.819369 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de5eef-daaa-48ba-8ae3-e1fe3f30af8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:48.819999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de5eef-daaa-48ba-8ae3-e1fe3f30af8d None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:48.820275 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de5eef-daaa-48ba-8ae3-e1fe3f30af8d None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:48.820941 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de5eef-daaa-48ba-8ae3-e1fe3f30af8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:48.827937 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8de5eef-daaa-48ba-8ae3-e1fe3f30af8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:48.828776 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-b1708de4-f39a-46bc-9014-80200ef519b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:48.828776 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:48.828776 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:48.828776 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:48.828776 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:48.828776 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:48.828776 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:48.828776 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:48.828776 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:48.828776 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:48.828776 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:48.828776 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:48.828776 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:19:48.828776 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:19:48.828776 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:19:48.828776 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:19:48.829814 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:48.829814 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:48.829814 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:48.829814 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:48.829814 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:48.829814 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:48.829814 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:48.829814 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:48.829814 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:48.829814 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:48.829814 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:48.829814 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:48.829814 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:48.829814 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:48.829814 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:48.829814 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:48.829814 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:19:48.830946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:19:48.830946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:48.830946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:48.831119 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8de5eef-daaa-48ba-8ae3-e1fe3f30af8d None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:48.831187 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 263/512] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:48.831187 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1708de4-f39a-46bc-9014-80200ef519b4 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:48.843764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-e8de5eef-daaa-48ba-8ae3-e1fe3f30af8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:19:48.843764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:48.843764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:48.843764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:48.843764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:48.843764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:48.843764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:48.843764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:48.843764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:48.843764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:48.843764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:48.843764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:48.843764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:19:48.843764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:19:48.843764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:19:48.844817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:19:48.844817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:48.844817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:48.844817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:48.844817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:48.844817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:48.844817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:48.844817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:48.844817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:48.844817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:48.844817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:48.844817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:48.844817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:48.844817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:48.844817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:48.844817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:48.844817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:48.845894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:19:48.845894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:19:48.845894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:19:48.845894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:19:48.845894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:19:48.845894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:19:48.845894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:19:48.845894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:48.845894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 250/513] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:19:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:48.845894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de5eef-daaa-48ba-8ae3-e1fe3f30af8d None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:48.894128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:48.933975 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:48.940410 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1708de4-f39a-46bc-9014-80200ef519b4 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:48.944696 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de5eef-daaa-48ba-8ae3-e1fe3f30af8d None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:48.974167 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5de2e-3e9e-4e28-be07-dc2b0f612696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:48.974765 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5de2e-3e9e-4e28-be07-dc2b0f612696 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:48.975063 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5de2e-3e9e-4e28-be07-dc2b0f612696 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:48.975331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5de2e-3e9e-4e28-be07-dc2b0f612696 None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:48.975961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf5de2e-3e9e-4e28-be07-dc2b0f612696 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:48.976257 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf5de2e-3e9e-4e28-be07-dc2b0f612696 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:48.977609 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf5de2e-3e9e-4e28-be07-dc2b0f612696 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:48.978511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd04a2b2-409d-43c7-8da7-dfac56b92ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:48.979130 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04a2b2-409d-43c7-8da7-dfac56b92ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:48.979406 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04a2b2-409d-43c7-8da7-dfac56b92ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:48.979677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04a2b2-409d-43c7-8da7-dfac56b92ce1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:48.987064 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd04a2b2-409d-43c7-8da7-dfac56b92ce1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:48.988477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd04a2b2-409d-43c7-8da7-dfac56b92ce1 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:48.988694 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 264/514] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:19:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:48.989471 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5de2e-3e9e-4e28-be07-dc2b0f612696 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:49.004105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-bd04a2b2-409d-43c7-8da7-dfac56b92ce1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 10:19:49.004105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:49.004105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:49.004105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:49.004105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:49.004105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:49.004105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:49.004105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:49.004105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:49.004105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:49.004105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:49.004105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:49.004105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:19:49.004105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:19:49.004105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:19:49.004105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:19:49.005036 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:49.005036 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:49.005036 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:19:49.005036 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:19:49.005036 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:19:49.005036 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:19:49.005036 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:49.005036 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:49.005036 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:49.005036 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:49.005036 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:49.005036 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:49.005036 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:49.005036 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:49.005036 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:49.005036 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:49.005036 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:49.005036 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:49.005946 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:49.005946 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:49.005946 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:19:49.005946 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:19:49.005946 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:19:49.005946 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:19:49.005946 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:19:49.005946 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:19:49.005946 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 10:19:49.005946 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:49.006957 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 251/515] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:19:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:49.007161 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd04a2b2-409d-43c7-8da7-dfac56b92ce1 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:49.089515 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5de2e-3e9e-4e28-be07-dc2b0f612696 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:49.110629 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd04a2b2-409d-43c7-8da7-dfac56b92ce1 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:49.120710 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:49.122843 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236b129-e0e1-4111-a7f6-645657eeb3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:49.123435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236b129-e0e1-4111-a7f6-645657eeb3b9 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:49.123732 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236b129-e0e1-4111-a7f6-645657eeb3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:49.123996 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236b129-e0e1-4111-a7f6-645657eeb3b9 None admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:49.124725 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236b129-e0e1-4111-a7f6-645657eeb3b9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:49.125010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236b129-e0e1-4111-a7f6-645657eeb3b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:49.126456 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236b129-e0e1-4111-a7f6-645657eeb3b9 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:49.145499 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd2838-32b3-48c3-a33a-50d00ecb118f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:49.146061 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd2838-32b3-48c3-a33a-50d00ecb118f None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:49.146339 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd2838-32b3-48c3-a33a-50d00ecb118f None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:49.146608 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd2838-32b3-48c3-a33a-50d00ecb118f None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:49.147240 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd2838-32b3-48c3-a33a-50d00ecb118f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:49.147538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd2838-32b3-48c3-a33a-50d00ecb118f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:49.148905 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd2838-32b3-48c3-a33a-50d00ecb118f None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:49.160547 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 252/516] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:19:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:49.161134 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd2838-32b3-48c3-a33a-50d00ecb118f None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:49.162178 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 265/517] 10.222.0.49 () {56 vars in 1137 bytes} [Mon Sep 26 10:19:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:49.162995 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236b129-e0e1-4111-a7f6-645657eeb3b9 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:49.178391 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:49.261824 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd2838-32b3-48c3-a33a-50d00ecb118f None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:49.264096 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236b129-e0e1-4111-a7f6-645657eeb3b9 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:49.295855 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5441d9c-11e6-46e6-bcc5-d560bb2ceca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:49.296204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e48853-1af1-45af-be05-fd2514dff599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:49.296448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5441d9c-11e6-46e6-bcc5-d560bb2ceca8 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:49.296726 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5441d9c-11e6-46e6-bcc5-d560bb2ceca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:49.296846 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e48853-1af1-45af-be05-fd2514dff599 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:49.296998 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5441d9c-11e6-46e6-bcc5-d560bb2ceca8 None admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:49.297122 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e48853-1af1-45af-be05-fd2514dff599 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:49.297390 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e48853-1af1-45af-be05-fd2514dff599 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:49.297679 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5441d9c-11e6-46e6-bcc5-d560bb2ceca8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:49.297982 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5441d9c-11e6-46e6-bcc5-d560bb2ceca8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:49.299456 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5441d9c-11e6-46e6-bcc5-d560bb2ceca8 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:49.304228 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e48853-1af1-45af-be05-fd2514dff599 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:49.305640 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e48853-1af1-45af-be05-fd2514dff599 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:49.316985 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 253/518] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:19:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:49.317772 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5441d9c-11e6-46e6-bcc5-d560bb2ceca8 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:49.318602 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-83e48853-1af1-45af-be05-fd2514dff599 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 26 10:19:49.318602 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:49.318602 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:49.318602 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:49.318602 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:49.318602 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:49.318602 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:49.318602 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:49.318602 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:49.318602 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:49.318602 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:49.318602 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:49.318602 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:19:49.318602 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:19:49.318602 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:19:49.318602 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:19:49.319472 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:49.319472 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:49.319472 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:19:49.319472 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:19:49.319472 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:19:49.319472 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:19:49.319472 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:49.319472 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:49.319472 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:49.319472 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:49.319472 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:49.319472 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:49.319472 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:49.319472 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:49.319472 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:49.319472 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:49.319472 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:49.319472 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:49.320344 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:49.320344 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:49.320344 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:19:49.320344 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:19:49.320344 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:19:49.320344 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:19:49.320344 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:19:49.320344 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:19:49.320344 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 26 10:19:49.320344 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:49.320344 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 266/519] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 10:19:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:49.320888 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e48853-1af1-45af-be05-fd2514dff599 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:49.349929 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:49.395786 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:49.419704 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5441d9c-11e6-46e6-bcc5-d560bb2ceca8 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:49.421611 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e48853-1af1-45af-be05-fd2514dff599 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:49.453702 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf021636-833e-478f-97f5-3682c433398c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:49.454329 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf021636-833e-478f-97f5-3682c433398c None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:49.454608 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf021636-833e-478f-97f5-3682c433398c None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:49.454879 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf021636-833e-478f-97f5-3682c433398c None admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:49.455270 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f88025-bc7d-4ef7-adee-5841c3e50443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:49.455853 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf021636-833e-478f-97f5-3682c433398c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:49.456147 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f88025-bc7d-4ef7-adee-5841c3e50443 None admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:49.456234 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf021636-833e-478f-97f5-3682c433398c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:49.456426 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f88025-bc7d-4ef7-adee-5841c3e50443 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:49.456700 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f88025-bc7d-4ef7-adee-5841c3e50443 None admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:49.457344 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f88025-bc7d-4ef7-adee-5841c3e50443 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:49.457566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf021636-833e-478f-97f5-3682c433398c None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:49.458806 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f88025-bc7d-4ef7-adee-5841c3e50443 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:49.475036 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 254/520] 10.222.0.49 () {56 vars in 1137 bytes} [Mon Sep 26 10:19:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:49.475891 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf021636-833e-478f-97f5-3682c433398c None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:49.479399 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 267/521] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 10:19:49 2022] POST /identity/v3/users => generated 247 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:19:49.495416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f88025-bc7d-4ef7-adee-5841c3e50443 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:49.585472 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:49.600773 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f88025-bc7d-4ef7-adee-5841c3e50443 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:49.600981 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf021636-833e-478f-97f5-3682c433398c None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:49.634817 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:49.640232 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40130eb1-3632-430b-ab51-89ef16fef163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:49.640820 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40130eb1-3632-430b-ab51-89ef16fef163 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:49.641102 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40130eb1-3632-430b-ab51-89ef16fef163 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:49.641408 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40130eb1-3632-430b-ab51-89ef16fef163 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:49.642471 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1786dc-cb89-43f4-aa91-5bba2afefe9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:49.643444 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1786dc-cb89-43f4-aa91-5bba2afefe9e None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:49.643938 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1786dc-cb89-43f4-aa91-5bba2afefe9e None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:49.644674 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1786dc-cb89-43f4-aa91-5bba2afefe9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:49.647058 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1786dc-cb89-43f4-aa91-5bba2afefe9e None admin] RBAC: Adding query filter params (role.id=d36e58344b604bcda1cee49065e8497e, scope.project.id=cff3a6011f4c4893bddffdad3d69f3df, user.id=cee41014d5d043888c6b399710bccab4) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:49.647314 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1786dc-cb89-43f4-aa91-5bba2afefe9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:49.648796 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1786dc-cb89-43f4-aa91-5bba2afefe9e None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:49.655406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 268/522] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:19:49 2022] GET /identity/v3/role_assignments?user.id=cee41014d5d043888c6b399710bccab4&scope.project.id=cff3a6011f4c4893bddffdad3d69f3df&role.id=d36e58344b604bcda1cee49065e8497e => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:49.656219 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1786dc-cb89-43f4-aa91-5bba2afefe9e None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:49.729340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40130eb1-3632-430b-ab51-89ef16fef163 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:49.731096 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40130eb1-3632-430b-ab51-89ef16fef163 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:49.801282 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1786dc-cb89-43f4-aa91-5bba2afefe9e None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:49.839855 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6305fac0-8b0d-4130-acca-d71469765e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:49.840768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305fac0-8b0d-4130-acca-d71469765e2a None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:49.841252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305fac0-8b0d-4130-acca-d71469765e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:49.841662 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305fac0-8b0d-4130-acca-d71469765e2a None admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:49.842556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6305fac0-8b0d-4130-acca-d71469765e2a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:49.843016 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6305fac0-8b0d-4130-acca-d71469765e2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:49.844568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6305fac0-8b0d-4130-acca-d71469765e2a None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:49.850496 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:49.866653 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 269/523] 10.222.0.49 () {56 vars in 1139 bytes} [Mon Sep 26 10:19:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:49.867594 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6305fac0-8b0d-4130-acca-d71469765e2a None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:49.868650 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 255/524] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:19:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 10:19:49.873762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40130eb1-3632-430b-ab51-89ef16fef163 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:49.929288 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:49.983839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6305fac0-8b0d-4130-acca-d71469765e2a None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:49.999368 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40130eb1-3632-430b-ab51-89ef16fef163 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:50.023973 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fe915-9ea4-411d-bfe1-5a9cc96f9f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:50.024676 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fe915-9ea4-411d-bfe1-5a9cc96f9f8e None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:50.025027 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fe915-9ea4-411d-bfe1-5a9cc96f9f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:50.025373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fe915-9ea4-411d-bfe1-5a9cc96f9f8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:50.041845 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33518a97-82b2-400b-b21c-5167327d8cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:50.042748 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33518a97-82b2-400b-b21c-5167327d8cea None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:50.043033 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33518a97-82b2-400b-b21c-5167327d8cea None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:50.043308 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33518a97-82b2-400b-b21c-5167327d8cea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:50.104323 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656fe915-9ea4-411d-bfe1-5a9cc96f9f8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:50.105911 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656fe915-9ea4-411d-bfe1-5a9cc96f9f8e None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:50.120539 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:50.137454 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33518a97-82b2-400b-b21c-5167327d8cea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:50.139503 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33518a97-82b2-400b-b21c-5167327d8cea None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:50.170522 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:50.238273 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 270/525] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:19:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 10:19:50.239063 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fe915-9ea4-411d-bfe1-5a9cc96f9f8e None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:50.267374 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 256/526] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:19:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 10:19:50.268298 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33518a97-82b2-400b-b21c-5167327d8cea None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:50.367407 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:50.367899 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fe915-9ea4-411d-bfe1-5a9cc96f9f8e None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:50.382044 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33518a97-82b2-400b-b21c-5167327d8cea None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:50.405388 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb062772-8519-43e8-a9c4-9db7604c12c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:50.406048 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb062772-8519-43e8-a9c4-9db7604c12c7 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:50.406332 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb062772-8519-43e8-a9c4-9db7604c12c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:50.406601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb062772-8519-43e8-a9c4-9db7604c12c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:50.418350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb062772-8519-43e8-a9c4-9db7604c12c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:50.419851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb062772-8519-43e8-a9c4-9db7604c12c7 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:50.426623 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036d74c-80f4-41ae-bf45-da4772666b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:50.427371 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036d74c-80f4-41ae-bf45-da4772666b6d None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:50.427657 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036d74c-80f4-41ae-bf45-da4772666b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:50.427933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036d74c-80f4-41ae-bf45-da4772666b6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:50.435826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-eb062772-8519-43e8-a9c4-9db7604c12c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:50.435826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:50.435826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:50.435826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:50.435826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:50.435826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:50.435826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:50.435826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:50.435826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:50.435826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:50.435826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:50.435826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:50.435826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:19:50.435826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:19:50.435826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:19:50.435826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:19:50.437863 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:50.437863 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:50.437863 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:50.437863 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:50.437863 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:50.437863 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:50.437863 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:50.437863 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:50.437863 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:50.437863 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:50.437863 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:50.437863 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:50.437863 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:50.437863 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:50.437863 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:50.437863 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:50.437863 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:19:50.440024 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:19:50.440024 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:50.440024 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:50.440024 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 271/527] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:50.440024 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb062772-8519-43e8-a9c4-9db7604c12c7 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:50.440776 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036d74c-80f4-41ae-bf45-da4772666b6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:50.440776 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036d74c-80f4-41ae-bf45-da4772666b6d None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:50.445332 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:50.459118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-d036d74c-80f4-41ae-bf45-da4772666b6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:50.459118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:50.459118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:50.459118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:50.459118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:50.459118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:50.459118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:50.459118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:50.459118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:50.459118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:50.459118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:50.459118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:50.459118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:19:50.459118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:19:50.459118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:19:50.459118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:19:50.461243 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:50.461243 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:50.461243 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:50.461243 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:50.461243 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:50.461243 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:50.461243 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:50.461243 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:50.461243 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:50.461243 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:50.461243 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:50.461243 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:50.461243 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:50.461243 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:50.461243 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:50.461243 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:50.461243 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:19:50.463272 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:19:50.463272 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:50.463272 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:50.470058 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 257/528] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:50.470058 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036d74c-80f4-41ae-bf45-da4772666b6d None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:50.555046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb062772-8519-43e8-a9c4-9db7604c12c7 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:50.574626 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036d74c-80f4-41ae-bf45-da4772666b6d None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:50.593238 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b3a83-7ef6-4587-a44f-13dce5ef08f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:50.593870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b3a83-7ef6-4587-a44f-13dce5ef08f0 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:50.594177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b3a83-7ef6-4587-a44f-13dce5ef08f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:50.594451 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b3a83-7ef6-4587-a44f-13dce5ef08f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:50.607063 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b3a83-7ef6-4587-a44f-13dce5ef08f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:50.608548 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b3a83-7ef6-4587-a44f-13dce5ef08f0 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:50.611246 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56fe77-c146-47bb-8928-452771b3c4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:50.611869 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56fe77-c146-47bb-8928-452771b3c4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:50.612151 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56fe77-c146-47bb-8928-452771b3c4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:50.612426 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56fe77-c146-47bb-8928-452771b3c4f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:50.613079 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c56fe77-c146-47bb-8928-452771b3c4f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:50.613383 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c56fe77-c146-47bb-8928-452771b3c4f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:50.614833 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c56fe77-c146-47bb-8928-452771b3c4f8 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:50.625520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-649b3a83-7ef6-4587-a44f-13dce5ef08f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:50.625520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:50.625520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:50.625520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:50.625520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:50.625520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:50.625520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:50.625520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:50.625520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:50.625520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:50.625520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:50.625520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:50.625520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:19:50.625520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:19:50.625520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:19:50.625520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:19:50.627415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:50.627415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:50.627415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:50.627415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:50.627415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:50.627415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:50.627415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:50.627415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:50.627415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:50.627415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:50.627415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:50.627415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:50.627415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:50.627415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:50.627415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:50.627415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:50.627415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:19:50.629074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:19:50.629074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:50.629074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:50.629360 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:50.629475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 272/529] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:50.629475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b3a83-7ef6-4587-a44f-13dce5ef08f0 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:50.634852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 258/530] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:19:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:50.642957 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56fe77-c146-47bb-8928-452771b3c4f8 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:50.696000 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:50.745525 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b3a83-7ef6-4587-a44f-13dce5ef08f0 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:50.746652 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56fe77-c146-47bb-8928-452771b3c4f8 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:50.782973 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd54d3-3db7-4605-97af-7b9f0b9468ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:50.783673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e619b1b-3916-45fb-9e21-3c771e37392b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:50.783858 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd54d3-3db7-4605-97af-7b9f0b9468ba None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:50.784073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd54d3-3db7-4605-97af-7b9f0b9468ba None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:50.784269 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e619b1b-3916-45fb-9e21-3c771e37392b None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:50.784509 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd54d3-3db7-4605-97af-7b9f0b9468ba None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:50.784657 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e619b1b-3916-45fb-9e21-3c771e37392b None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:50.784805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e619b1b-3916-45fb-9e21-3c771e37392b None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:50.785386 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd54d3-3db7-4605-97af-7b9f0b9468ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:50.785527 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e619b1b-3916-45fb-9e21-3c771e37392b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:50.785726 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e619b1b-3916-45fb-9e21-3c771e37392b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:50.785862 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd54d3-3db7-4605-97af-7b9f0b9468ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:50.787125 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e619b1b-3916-45fb-9e21-3c771e37392b None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:50.787365 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd54d3-3db7-4605-97af-7b9f0b9468ba None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:50.806803 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 259/532] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:19:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:50.806803 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd54d3-3db7-4605-97af-7b9f0b9468ba None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:50.807701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 273/532] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:19:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:50.814042 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e619b1b-3916-45fb-9e21-3c771e37392b None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:50.868998 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:50.914287 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd54d3-3db7-4605-97af-7b9f0b9468ba None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:50.920789 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e619b1b-3916-45fb-9e21-3c771e37392b None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:50.939494 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:50.950178 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be9e97-f780-4bbe-a4f0-2df6d7abf5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:50.951421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be9e97-f780-4bbe-a4f0-2df6d7abf5bf None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:50.951703 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be9e97-f780-4bbe-a4f0-2df6d7abf5bf None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:50.951978 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be9e97-f780-4bbe-a4f0-2df6d7abf5bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:50.959470 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be9e97-f780-4bbe-a4f0-2df6d7abf5bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:50.960704 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515069e9-795e-45bf-b6f0-5a5d3ab8aca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:50.961025 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be9e97-f780-4bbe-a4f0-2df6d7abf5bf None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:50.961493 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515069e9-795e-45bf-b6f0-5a5d3ab8aca7 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:50.961778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515069e9-795e-45bf-b6f0-5a5d3ab8aca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:50.962071 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515069e9-795e-45bf-b6f0-5a5d3ab8aca7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:50.969255 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515069e9-795e-45bf-b6f0-5a5d3ab8aca7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:50.970924 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515069e9-795e-45bf-b6f0-5a5d3ab8aca7 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:50.972669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-08be9e97-f780-4bbe-a4f0-2df6d7abf5bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:19:50.972669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:50.972669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:50.972669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:50.972669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:50.972669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:50.972669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:50.972669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:50.972669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:50.972669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:50.972669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:50.972669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:50.972669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:19:50.972669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:19:50.972669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:19:50.974579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:19:50.974579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:50.974579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:50.974579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:50.974579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:50.974579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:50.974579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:50.974579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:50.974579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:50.974579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:50.974579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:50.974579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:50.974579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:50.974579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:50.974579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:50.974579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:50.974579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:50.976579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:19:50.976579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:19:50.976579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:19:50.976579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:19:50.976579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:19:50.976579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:19:50.976579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:19:50.976579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:50.986045 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 260/533] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:19:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:50.986045 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be9e97-f780-4bbe-a4f0-2df6d7abf5bf None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:50.986435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-515069e9-795e-45bf-b6f0-5a5d3ab8aca7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:19:50.986435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:50.986435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:50.986435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:50.986435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:50.986435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:50.986435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:50.986435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:50.986435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:50.986435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:50.986435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:50.986435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:50.986435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:19:50.986435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:19:50.986435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:19:50.988221 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:19:50.988221 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:50.988221 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:50.988221 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:50.988221 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:50.988221 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:50.988221 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:50.988221 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:50.988221 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:50.988221 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:50.988221 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:50.988221 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:50.988221 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:50.988221 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:50.988221 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:50.988221 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:50.988221 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:50.990264 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:19:50.990264 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:19:50.990264 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:19:50.990264 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:19:50.990264 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:19:50.990264 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:19:50.990264 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:19:50.990264 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:50.990264 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 274/534] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:19:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:50.990264 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515069e9-795e-45bf-b6f0-5a5d3ab8aca7 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:51.088570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be9e97-f780-4bbe-a4f0-2df6d7abf5bf None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:51.108991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:51.116177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515069e9-795e-45bf-b6f0-5a5d3ab8aca7 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:51.124855 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca2ab0-2146-4b19-bb23-c6662d85e16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:51.125848 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca2ab0-2146-4b19-bb23-c6662d85e16b None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:51.126164 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca2ab0-2146-4b19-bb23-c6662d85e16b None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:51.126442 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca2ab0-2146-4b19-bb23-c6662d85e16b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:51.133430 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca2ab0-2146-4b19-bb23-c6662d85e16b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:51.134966 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca2ab0-2146-4b19-bb23-c6662d85e16b None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:51.146736 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-2cca2ab0-2146-4b19-bb23-c6662d85e16b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:19:51.146736 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:51.146736 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:51.146736 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:51.146736 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:51.146736 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:51.146736 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:51.146736 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:51.146736 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:51.146736 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:51.146736 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:51.146736 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:51.146736 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:19:51.146736 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:19:51.146736 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:19:51.148626 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:19:51.148626 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:51.148626 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:51.148626 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:51.148626 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:51.148626 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:51.148626 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:51.148626 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:51.148626 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:51.148626 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:51.148626 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:51.148626 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:51.148626 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:51.148626 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:51.148626 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:51.148626 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:51.148626 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:51.150766 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:19:51.150766 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:19:51.150766 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:19:51.150766 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:19:51.150766 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:19:51.150766 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:19:51.150766 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:19:51.150766 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:51.150766 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 261/535] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:19:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:51.150766 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca2ab0-2146-4b19-bb23-c6662d85e16b None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:51.159126 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5cb8e8-12bd-458d-a122-fb2c5383a799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:51.159738 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5cb8e8-12bd-458d-a122-fb2c5383a799 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:51.160112 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5cb8e8-12bd-458d-a122-fb2c5383a799 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:51.160389 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5cb8e8-12bd-458d-a122-fb2c5383a799 None admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:51.161083 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5cb8e8-12bd-458d-a122-fb2c5383a799 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:51.161471 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5cb8e8-12bd-458d-a122-fb2c5383a799 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:51.163045 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5cb8e8-12bd-458d-a122-fb2c5383a799 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:51.188882 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 275/536] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:19:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:51.189881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5cb8e8-12bd-458d-a122-fb2c5383a799 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:51.242867 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:51.277971 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca2ab0-2146-4b19-bb23-c6662d85e16b None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:51.314011 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def1771-1612-4631-8f4f-bced7f781b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:51.314759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def1771-1612-4631-8f4f-bced7f781b05 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:51.315131 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def1771-1612-4631-8f4f-bced7f781b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:51.315471 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def1771-1612-4631-8f4f-bced7f781b05 None admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:51.316164 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def1771-1612-4631-8f4f-bced7f781b05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:51.316469 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def1771-1612-4631-8f4f-bced7f781b05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:51.318081 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def1771-1612-4631-8f4f-bced7f781b05 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:51.332778 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 262/537] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:19:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:51.333625 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def1771-1612-4631-8f4f-bced7f781b05 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:51.337830 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5cb8e8-12bd-458d-a122-fb2c5383a799 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:51.377628 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1ea5e-f408-4b85-a48c-45e2b88e3311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:51.378187 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1ea5e-f408-4b85-a48c-45e2b88e3311 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:51.378466 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1ea5e-f408-4b85-a48c-45e2b88e3311 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:51.378737 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1ea5e-f408-4b85-a48c-45e2b88e3311 None admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:51.379428 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1ea5e-f408-4b85-a48c-45e2b88e3311 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:51.379732 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1ea5e-f408-4b85-a48c-45e2b88e3311 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:51.381024 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:51.381221 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1ea5e-f408-4b85-a48c-45e2b88e3311 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:51.394446 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 276/538] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:19:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:51.398842 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1ea5e-f408-4b85-a48c-45e2b88e3311 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:51.461795 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def1771-1612-4631-8f4f-bced7f781b05 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:51.511753 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:51.512305 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa1a3a5-e3fb-4e4e-a202-2d57362803e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:51.513992 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1a3a5-e3fb-4e4e-a202-2d57362803e6 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:51.514367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1a3a5-e3fb-4e4e-a202-2d57362803e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:51.514640 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1a3a5-e3fb-4e4e-a202-2d57362803e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:51.515377 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa1a3a5-e3fb-4e4e-a202-2d57362803e6 None admin] RBAC: Adding query filter params (role.id=d36e58344b604bcda1cee49065e8497e, scope.project.id=cff3a6011f4c4893bddffdad3d69f3df, user.id=41b242fbe2504f8cac26ce4ff46f5390) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:51.515684 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa1a3a5-e3fb-4e4e-a202-2d57362803e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:51.517375 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa1a3a5-e3fb-4e4e-a202-2d57362803e6 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:51.524506 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1ea5e-f408-4b85-a48c-45e2b88e3311 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:51.530118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 263/539] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:19:51 2022] GET /identity/v3/role_assignments?user.id=41b242fbe2504f8cac26ce4ff46f5390&scope.project.id=cff3a6011f4c4893bddffdad3d69f3df&role.id=d36e58344b604bcda1cee49065e8497e => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:51.531148 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa1a3a5-e3fb-4e4e-a202-2d57362803e6 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:51.570978 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b79d601-b668-4cf6-aaff-bf904623a08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:51.571661 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79d601-b668-4cf6-aaff-bf904623a08b None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:51.571942 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79d601-b668-4cf6-aaff-bf904623a08b None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:51.572215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79d601-b668-4cf6-aaff-bf904623a08b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:51.573154 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b79d601-b668-4cf6-aaff-bf904623a08b None admin] RBAC: Adding query filter params (role.id=d36e58344b604bcda1cee49065e8497e, scope.project.id=cff3a6011f4c4893bddffdad3d69f3df, user.id=f841bf31c43b4333a3d834d38b5197e1) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:51.573560 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b79d601-b668-4cf6-aaff-bf904623a08b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:51.575040 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b79d601-b668-4cf6-aaff-bf904623a08b None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:51.586488 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 277/540] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:19:51 2022] GET /identity/v3/role_assignments?user.id=f841bf31c43b4333a3d834d38b5197e1&scope.project.id=cff3a6011f4c4893bddffdad3d69f3df&role.id=d36e58344b604bcda1cee49065e8497e => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:51.712930 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa1a3a5-e3fb-4e4e-a202-2d57362803e6 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:51.720527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:51.778006 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b96c1-ede5-4b16-bfbb-c759867fa9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:51.779239 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b96c1-ede5-4b16-bfbb-c759867fa9d3 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:51.779521 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b96c1-ede5-4b16-bfbb-c759867fa9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:51.779790 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b96c1-ede5-4b16-bfbb-c759867fa9d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:51.780530 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526b96c1-ede5-4b16-bfbb-c759867fa9d3 None admin] RBAC: Adding query filter params (role.id=d36e58344b604bcda1cee49065e8497e, scope.project.id=cff3a6011f4c4893bddffdad3d69f3df, user.id=feddae5ba018426697854931a55104e3) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:51.780832 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526b96c1-ede5-4b16-bfbb-c759867fa9d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:51.782376 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526b96c1-ede5-4b16-bfbb-c759867fa9d3 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:51.795195 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 264/541] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:19:51 2022] GET /identity/v3/role_assignments?user.id=feddae5ba018426697854931a55104e3&scope.project.id=cff3a6011f4c4893bddffdad3d69f3df&role.id=d36e58344b604bcda1cee49065e8497e => generated 259 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:51.795509 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e9861-588a-4060-be6c-dd3ee0d1e355 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:51.795509 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e9861-588a-4060-be6c-dd3ee0d1e355 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:51.795819 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e9861-588a-4060-be6c-dd3ee0d1e355 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:51.798410 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 278/542] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:51.806996 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6171c-1b9b-4dd5-bea3-e3dfc58e5233 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:51.806996 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6171c-1b9b-4dd5-bea3-e3dfc58e5233 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:51.806996 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6171c-1b9b-4dd5-bea3-e3dfc58e5233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:51.828837 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-31b6171c-1b9b-4dd5-bea3-e3dfc58e5233 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:51.830189 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-31b6171c-1b9b-4dd5-bea3-e3dfc58e5233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:51.915437 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 265/543] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:52.135983 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a0d67-9104-4c95-be80-465e58ecf8dd None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:52.136828 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a0d67-9104-4c95-be80-465e58ecf8dd None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:52.137278 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a0d67-9104-4c95-be80-465e58ecf8dd None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:52.139705 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 279/544] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:52.144601 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84c853-4f39-4c40-9acb-373928b85931 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:52.144887 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84c853-4f39-4c40-9acb-373928b85931 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:52.145161 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84c853-4f39-4c40-9acb-373928b85931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:52.171029 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-cb84c853-4f39-4c40-9acb-373928b85931 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:52.172259 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-cb84c853-4f39-4c40-9acb-373928b85931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:52.243814 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 266/545] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:52.322915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a0d67-9104-4c95-be80-465e58ecf8dd None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:52.455957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a0d67-9104-4c95-be80-465e58ecf8dd None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:52.504805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5c170-182b-42c8-a2cf-9616b6ba673a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:52.505794 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5c170-182b-42c8-a2cf-9616b6ba673a None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:52.506290 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5c170-182b-42c8-a2cf-9616b6ba673a None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:52.506758 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5c170-182b-42c8-a2cf-9616b6ba673a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:52.526070 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb84c853-4f39-4c40-9acb-373928b85931 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:52.527056 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea5c170-182b-42c8-a2cf-9616b6ba673a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:52.531954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-7ea5c170-182b-42c8-a2cf-9616b6ba673a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:19:52.531954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:52.531954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:52.531954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:52.531954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:52.531954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:52.531954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:52.531954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:52.531954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:52.531954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:52.531954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:52.531954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:52.531954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:52.531954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:52.531954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:52.531954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:52.532826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:52.532826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:52.532826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:52.532826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:52.532826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:52.532826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:52.532826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:52.532826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:52.532826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:52.532826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:52.532826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:52.532826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:52.532826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:52.532826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:52.532826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:52.532826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:52.534233 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:52.534233 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:52.534233 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:52.534233 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:52.534233 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:52.534233 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:52.534233 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:52.534233 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:52.534233 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:52.534233 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:52.534233 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:19:52.534233 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:52.534233 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 280/546] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 10:19:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:52.538606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5c170-182b-42c8-a2cf-9616b6ba673a None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:52.554070 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:52.582397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:52.661914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb84c853-4f39-4c40-9acb-373928b85931 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:52.691905 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5c170-182b-42c8-a2cf-9616b6ba673a None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:52.703402 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84610d53-a392-48b9-a547-276dd8f62d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:52.704102 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84610d53-a392-48b9-a547-276dd8f62d2a None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:52.704480 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84610d53-a392-48b9-a547-276dd8f62d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:52.704757 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84610d53-a392-48b9-a547-276dd8f62d2a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:52.716486 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84610d53-a392-48b9-a547-276dd8f62d2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:52.721716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-84610d53-a392-48b9-a547-276dd8f62d2a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:19:52.721716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:52.721716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:52.721716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:52.721716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:52.721716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:52.721716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:52.721716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:52.721716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:52.721716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:52.721716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:52.721716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:52.721716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:52.721716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:52.721716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:52.721716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:52.722655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:52.722655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:52.722655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:52.722655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:52.722655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:52.722655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:52.722655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:52.722655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:52.722655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:52.722655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:52.722655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:52.722655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:52.722655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:52.722655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:52.722655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:52.722655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:52.723836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:52.723836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:52.723836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:52.723836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:52.723836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:52.723836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:52.723836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:52.723836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:52.723836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:52.723836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:52.723836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:19:52.723836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:52.723836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 267/547] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 10:19:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:52.728983 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84610d53-a392-48b9-a547-276dd8f62d2a None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:52.729907 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef42e8-4255-4d4b-aed2-b55a0c65c30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:52.730570 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef42e8-4255-4d4b-aed2-b55a0c65c30f None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:52.730846 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef42e8-4255-4d4b-aed2-b55a0c65c30f None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:52.731119 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef42e8-4255-4d4b-aed2-b55a0c65c30f None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:52.732354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ef42e8-4255-4d4b-aed2-b55a0c65c30f None admin] RBAC: Adding query filter params (name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:52.732682 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ef42e8-4255-4d4b-aed2-b55a0c65c30f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:52.735652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ef42e8-4255-4d4b-aed2-b55a0c65c30f None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:52.745446 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 281/548] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 10:19:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:52.752546 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef42e8-4255-4d4b-aed2-b55a0c65c30f None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:52.862539 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:52.881181 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef42e8-4255-4d4b-aed2-b55a0c65c30f None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:52.914053 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:52.918825 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb59ef5-9bac-4562-93f3-36f4434f5c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:52.920813 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb59ef5-9bac-4562-93f3-36f4434f5c7e None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:52.921086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb59ef5-9bac-4562-93f3-36f4434f5c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:52.921353 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb59ef5-9bac-4562-93f3-36f4434f5c7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:52.933381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb59ef5-9bac-4562-93f3-36f4434f5c7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:52.934892 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb59ef5-9bac-4562-93f3-36f4434f5c7e None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:52.941161 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84610d53-a392-48b9-a547-276dd8f62d2a None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:52.952769 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-acb59ef5-9bac-4562-93f3-36f4434f5c7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:52.952769 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:52.952769 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:52.952769 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:52.952769 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:52.952769 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:52.952769 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:52.952769 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:52.952769 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:52.952769 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:52.952769 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:52.952769 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:52.952769 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:19:52.952769 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:19:52.952769 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:19:52.952769 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:19:52.955923 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:52.955923 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:52.955923 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:52.955923 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:52.955923 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:52.955923 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:52.955923 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:52.955923 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:52.955923 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:52.955923 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:52.955923 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:52.955923 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:52.955923 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:52.955923 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:52.955923 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:52.955923 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:52.955923 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:19:52.957928 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:19:52.957928 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:52.957928 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:52.957928 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 282/549] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:19:52.961363 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb59ef5-9bac-4562-93f3-36f4434f5c7e None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:52.984314 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe388d6-2721-49c1-a7e0-1ae366e04270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:52.984991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe388d6-2721-49c1-a7e0-1ae366e04270 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:52.985387 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe388d6-2721-49c1-a7e0-1ae366e04270 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:52.985760 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe388d6-2721-49c1-a7e0-1ae366e04270 None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:52.986652 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe388d6-2721-49c1-a7e0-1ae366e04270 None admin] RBAC: Adding query filter params (name=service) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:52.987055 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe388d6-2721-49c1-a7e0-1ae366e04270 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:52.988520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe388d6-2721-49c1-a7e0-1ae366e04270 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:52.997695 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 268/550] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 10:19:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:53.006334 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe388d6-2721-49c1-a7e0-1ae366e04270 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:53.134209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb59ef5-9bac-4562-93f3-36f4434f5c7e None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:53.162267 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe388d6-2721-49c1-a7e0-1ae366e04270 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:53.174737 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c07733-9670-40db-83f8-76a6c9690fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:53.175359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c07733-9670-40db-83f8-76a6c9690fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:53.175600 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c07733-9670-40db-83f8-76a6c9690fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:53.175869 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c07733-9670-40db-83f8-76a6c9690fa0 None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:53.176776 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c07733-9670-40db-83f8-76a6c9690fa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:53.177075 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c07733-9670-40db-83f8-76a6c9690fa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:53.179242 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c07733-9670-40db-83f8-76a6c9690fa0 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:53.180844 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:53.201653 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 283/551] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:19:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:53.211801 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c07733-9670-40db-83f8-76a6c9690fa0 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:53.229132 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1397750-6ea2-4eb5-9cda-3b5cfb422cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:53.229781 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1397750-6ea2-4eb5-9cda-3b5cfb422cb8 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:53.230191 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1397750-6ea2-4eb5-9cda-3b5cfb422cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:53.230465 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1397750-6ea2-4eb5-9cda-3b5cfb422cb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:53.251143 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1397750-6ea2-4eb5-9cda-3b5cfb422cb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:53.252694 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1397750-6ea2-4eb5-9cda-3b5cfb422cb8 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:53.281955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-a1397750-6ea2-4eb5-9cda-3b5cfb422cb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:53.281955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:53.281955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:53.281955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:53.281955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:53.281955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:53.281955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:53.281955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:53.281955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:53.281955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:53.281955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:53.281955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:53.281955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:19:53.281955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:19:53.281955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:19:53.281955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:19:53.284420 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:53.284420 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:53.284420 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:53.284420 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:53.284420 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:53.284420 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:53.284420 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:53.284420 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:53.284420 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:53.284420 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:53.284420 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:53.284420 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:53.284420 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:53.284420 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:53.284420 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:53.284420 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:53.284420 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:19:53.286671 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:19:53.286671 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:53.286671 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:53.290438 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 269/552] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:53.290438 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1397750-6ea2-4eb5-9cda-3b5cfb422cb8 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:53.295183 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:53.318656 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c07733-9670-40db-83f8-76a6c9690fa0 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:53.357806 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d11ab-35e7-4e45-965e-ec7ca658ec04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:53.358460 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d11ab-35e7-4e45-965e-ec7ca658ec04 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:53.358736 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d11ab-35e7-4e45-965e-ec7ca658ec04 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:53.359002 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d11ab-35e7-4e45-965e-ec7ca658ec04 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:53.366328 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d11ab-35e7-4e45-965e-ec7ca658ec04 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:53.367883 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d11ab-35e7-4e45-965e-ec7ca658ec04 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:53.379877 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-c97d11ab-35e7-4e45-965e-ec7ca658ec04 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 26 10:19:53.379877 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:53.379877 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:53.379877 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:53.379877 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:53.379877 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:53.379877 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:53.379877 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:53.379877 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:53.379877 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:53.379877 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:53.379877 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:53.379877 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:19:53.379877 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:19:53.379877 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:19:53.379877 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:19:53.382102 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:53.382102 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:53.382102 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:19:53.382102 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:19:53.382102 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:19:53.382102 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:19:53.382102 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:53.382102 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:53.382102 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:53.382102 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:53.382102 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:53.382102 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:53.382102 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:53.382102 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:53.382102 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:53.382102 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:53.382102 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:53.382102 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:53.384356 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:53.384356 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:53.384356 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:19:53.384356 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:19:53.384356 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:19:53.384356 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:19:53.384356 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:19:53.384356 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:19:53.384356 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 26 10:19:53.384356 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:53.384356 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 284/553] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:19:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:19:53.390602 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d11ab-35e7-4e45-965e-ec7ca658ec04 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:53.454465 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:53.501138 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d11ab-35e7-4e45-965e-ec7ca658ec04 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:53.552711 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1397750-6ea2-4eb5-9cda-3b5cfb422cb8 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:53.554603 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8825ebed-2985-4f20-b8d3-84535f2ff2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:53.555209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8825ebed-2985-4f20-b8d3-84535f2ff2db None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:53.555482 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8825ebed-2985-4f20-b8d3-84535f2ff2db None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:53.555749 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8825ebed-2985-4f20-b8d3-84535f2ff2db None admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:53.556476 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8825ebed-2985-4f20-b8d3-84535f2ff2db None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:53.556774 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8825ebed-2985-4f20-b8d3-84535f2ff2db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:53.559319 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8825ebed-2985-4f20-b8d3-84535f2ff2db None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:53.593432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 285/554] 10.222.0.49 () {56 vars in 1135 bytes} [Mon Sep 26 10:19:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:53.599918 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8825ebed-2985-4f20-b8d3-84535f2ff2db None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:53.629284 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a3a73-6d14-437b-bb0c-21b7ec8b8f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:53.629963 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a3a73-6d14-437b-bb0c-21b7ec8b8f30 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:53.630268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a3a73-6d14-437b-bb0c-21b7ec8b8f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:53.630538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a3a73-6d14-437b-bb0c-21b7ec8b8f30 None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:53.631188 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863a3a73-6d14-437b-bb0c-21b7ec8b8f30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:53.631483 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863a3a73-6d14-437b-bb0c-21b7ec8b8f30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:53.632843 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863a3a73-6d14-437b-bb0c-21b7ec8b8f30 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:53.658304 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 270/555] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:19:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:53.663841 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a3a73-6d14-437b-bb0c-21b7ec8b8f30 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:53.748132 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8825ebed-2985-4f20-b8d3-84535f2ff2db None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:53.762336 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:53.806749 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63aecce-6b5f-402f-9ef4-772cf550f77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:53.807393 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63aecce-6b5f-402f-9ef4-772cf550f77f None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:53.807659 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63aecce-6b5f-402f-9ef4-772cf550f77f None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:53.807932 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63aecce-6b5f-402f-9ef4-772cf550f77f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:53.826510 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:53.875840 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a3a73-6d14-437b-bb0c-21b7ec8b8f30 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:53.922134 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda79bc1-6f4a-45b7-86b8-508f2c86bdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:53.925311 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda79bc1-6f4a-45b7-86b8-508f2c86bdf4 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:53.925853 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda79bc1-6f4a-45b7-86b8-508f2c86bdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:53.926350 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda79bc1-6f4a-45b7-86b8-508f2c86bdf4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:53.937912 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda79bc1-6f4a-45b7-86b8-508f2c86bdf4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:53.939874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda79bc1-6f4a-45b7-86b8-508f2c86bdf4 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:53.960361 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-dda79bc1-6f4a-45b7-86b8-508f2c86bdf4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 10:19:53.960361 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:53.960361 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:53.960361 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:53.960361 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:53.960361 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:53.960361 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:53.960361 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:53.960361 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:53.960361 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:53.960361 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:53.960361 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:53.960361 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:19:53.960361 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:19:53.960361 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:19:53.960361 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:19:53.962268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:53.962268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:53.962268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:19:53.962268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:19:53.962268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:19:53.962268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:19:53.962268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:53.962268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:53.962268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:53.962268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:53.962268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:53.962268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:53.962268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:53.962268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:53.962268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:53.962268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:53.962268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:53.962268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:53.963192 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:53.963192 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:53.963192 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:19:53.963192 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:19:53.963192 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:19:53.963192 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:19:53.963192 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:19:53.963192 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:19:53.963192 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 10:19:53.963192 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:53.965642 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 271/556] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Sep 26 10:19:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:19:53.967223 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63aecce-6b5f-402f-9ef4-772cf550f77f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:53.968904 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63aecce-6b5f-402f-9ef4-772cf550f77f None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:53.971913 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda79bc1-6f4a-45b7-86b8-508f2c86bdf4 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:54.135742 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda79bc1-6f4a-45b7-86b8-508f2c86bdf4 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:54.166640 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 286/557] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:19:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 10:19:54.172947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63aecce-6b5f-402f-9ef4-772cf550f77f None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:54.188430 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:54.188715 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:54.213096 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c444b195-342b-4328-b7db-046c67c0dc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:54.213424 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c444b195-342b-4328-b7db-046c67c0dc4e None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:54.213626 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c444b195-342b-4328-b7db-046c67c0dc4e None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:54.213886 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c444b195-342b-4328-b7db-046c67c0dc4e None admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:54.214659 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c444b195-342b-4328-b7db-046c67c0dc4e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:54.214964 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c444b195-342b-4328-b7db-046c67c0dc4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:54.216449 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c444b195-342b-4328-b7db-046c67c0dc4e None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:54.265022 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 272/558] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Sep 26 10:19:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:54.265022 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522376c-2a02-4051-b590-fa1e1ca17e15 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:54.265022 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522376c-2a02-4051-b590-fa1e1ca17e15 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:54.265022 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522376c-2a02-4051-b590-fa1e1ca17e15 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:54.268046 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 273/559] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:54.269401 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f0dc7-1fd4-4c52-8411-538c43bbe04d None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:54.269682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f0dc7-1fd4-4c52-8411-538c43bbe04d None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:54.269955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f0dc7-1fd4-4c52-8411-538c43bbe04d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:54.290446 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-d76f0dc7-1fd4-4c52-8411-538c43bbe04d None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:54.291701 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-d76f0dc7-1fd4-4c52-8411-538c43bbe04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:54.296977 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63aecce-6b5f-402f-9ef4-772cf550f77f None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:54.339343 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f9f035-e248-407b-ae81-ccf4e8270973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:54.340363 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9f035-e248-407b-ae81-ccf4e8270973 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:54.340873 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9f035-e248-407b-ae81-ccf4e8270973 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:54.342000 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9f035-e248-407b-ae81-ccf4e8270973 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:54.356733 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f9f035-e248-407b-ae81-ccf4e8270973 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:54.359028 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f9f035-e248-407b-ae81-ccf4e8270973 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:54.380389 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 274/560] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:54.381225 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f0dc7-1fd4-4c52-8411-538c43bbe04d None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:54.381872 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-e7f9f035-e248-407b-ae81-ccf4e8270973 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:54.381872 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:54.381872 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:54.381872 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:54.381872 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:54.381872 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:54.381872 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:54.381872 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:54.381872 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:54.381872 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:54.381872 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:54.381872 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:54.381872 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:19:54.381872 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:19:54.381872 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:19:54.381872 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:19:54.383105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:54.383105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:54.383105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:54.383105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:54.383105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:54.383105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:54.383105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:54.383105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:54.383105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:54.383105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:54.383105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:54.383105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:54.383105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:54.383105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:54.383105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:54.383105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:54.383105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:19:54.383996 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:19:54.383996 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:54.383996 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:54.385969 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 287/561] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:54.392821 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f9f035-e248-407b-ae81-ccf4e8270973 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:54.476568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:54.502130 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f0dc7-1fd4-4c52-8411-538c43bbe04d None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:54.513958 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:54.520479 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f9f035-e248-407b-ae81-ccf4e8270973 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:54.537714 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412b043-b1a0-4df2-b917-c309651545d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:54.538358 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412b043-b1a0-4df2-b917-c309651545d0 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:54.538634 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412b043-b1a0-4df2-b917-c309651545d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:54.538912 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412b043-b1a0-4df2-b917-c309651545d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:54.549380 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412b043-b1a0-4df2-b917-c309651545d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:54.550911 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412b043-b1a0-4df2-b917-c309651545d0 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:54.555637 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b7d52-6542-422c-a025-69864ad7b0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:54.556233 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b7d52-6542-422c-a025-69864ad7b0f2 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:54.556508 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b7d52-6542-422c-a025-69864ad7b0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:54.556775 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b7d52-6542-422c-a025-69864ad7b0f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:54.557415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2b7d52-6542-422c-a025-69864ad7b0f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:54.557715 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2b7d52-6542-422c-a025-69864ad7b0f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:54.559096 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2b7d52-6542-422c-a025-69864ad7b0f2 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:54.566672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-3412b043-b1a0-4df2-b917-c309651545d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:54.566672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:54.566672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:54.566672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:54.566672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:54.566672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:54.566672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:54.566672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:54.566672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:54.566672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:54.566672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:54.566672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:54.566672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:19:54.566672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:19:54.566672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:19:54.566672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:19:54.567780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:54.567780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:54.567780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:54.567780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:54.567780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:54.567780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:54.567780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:54.567780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:54.567780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:54.567780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:54.567780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:54.567780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:54.567780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:54.567780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:54.567780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:54.567780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:54.567780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:19:54.568678 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:19:54.568678 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:54.568678 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:54.568678 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 275/562] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:54.569324 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fa101-23ee-43b2-90de-8c029a4d7848 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:54.569605 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fa101-23ee-43b2-90de-8c029a4d7848 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:54.569876 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1fa101-23ee-43b2-90de-8c029a4d7848 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:54.574526 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 276/563] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:54.574526 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1fd688-c4a3-4e3f-9afb-ce85c90d4340 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:54.574526 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1fd688-c4a3-4e3f-9afb-ce85c90d4340 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:54.574792 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1fd688-c4a3-4e3f-9afb-ce85c90d4340 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:54.577529 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 277/564] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:54.578256 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1fd688-c4a3-4e3f-9afb-ce85c90d4340 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:54.589708 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 288/565] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:19:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:54.589708 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc259b2-bd45-422c-91d4-ab62c4fd616a None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:54.589708 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc259b2-bd45-422c-91d4-ab62c4fd616a None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:54.589708 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc259b2-bd45-422c-91d4-ab62c4fd616a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:54.606111 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-0dc259b2-bd45-422c-91d4-ab62c4fd616a None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:54.607584 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-0dc259b2-bd45-422c-91d4-ab62c4fd616a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:54.678762 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 289/566] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:54.679966 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465c8c7-5bf3-411c-9359-31b9c927a9a7 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:54.680256 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465c8c7-5bf3-411c-9359-31b9c927a9a7 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:54.680527 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465c8c7-5bf3-411c-9359-31b9c927a9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:54.695208 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1fd688-c4a3-4e3f-9afb-ce85c90d4340 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:54.698119 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-b465c8c7-5bf3-411c-9359-31b9c927a9a7 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:54.699291 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-b465c8c7-5bf3-411c-9359-31b9c927a9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:54.737920 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c916c20-731a-4dca-a9e0-a4340fb0f35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:54.738746 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c916c20-731a-4dca-a9e0-a4340fb0f35b None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:54.739212 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c916c20-731a-4dca-a9e0-a4340fb0f35b None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:54.740079 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c916c20-731a-4dca-a9e0-a4340fb0f35b None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:54.740884 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c916c20-731a-4dca-a9e0-a4340fb0f35b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:54.741339 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c916c20-731a-4dca-a9e0-a4340fb0f35b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:54.743548 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c916c20-731a-4dca-a9e0-a4340fb0f35b None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:54.749241 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:54.764090 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 278/567] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:19:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:54.764090 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c916c20-731a-4dca-a9e0-a4340fb0f35b None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:54.767997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 290/568] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:54.773979 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465c8c7-5bf3-411c-9359-31b9c927a9a7 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:54.792125 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:54.888555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c916c20-731a-4dca-a9e0-a4340fb0f35b None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:54.905288 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465c8c7-5bf3-411c-9359-31b9c927a9a7 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:54.926441 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da41d26a-266e-44a6-b324-1528e8b4f77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:54.927302 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41d26a-266e-44a6-b324-1528e8b4f77a None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:54.927730 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41d26a-266e-44a6-b324-1528e8b4f77a None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:54.928223 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41d26a-266e-44a6-b324-1528e8b4f77a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:54.935762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da41d26a-266e-44a6-b324-1528e8b4f77a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:54.937528 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da41d26a-266e-44a6-b324-1528e8b4f77a None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:54.942800 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5966d8-9c54-419d-98db-32558366fc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:54.943673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5966d8-9c54-419d-98db-32558366fc76 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:54.943971 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5966d8-9c54-419d-98db-32558366fc76 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:54.944331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5966d8-9c54-419d-98db-32558366fc76 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:54.950144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-da41d26a-266e-44a6-b324-1528e8b4f77a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:19:54.950144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:54.950144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:54.950144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:54.950144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:54.950144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:54.950144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:54.950144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:54.950144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:54.950144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:54.950144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:54.950144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:54.950144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:19:54.950144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:19:54.950144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:19:54.951555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:19:54.951555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:54.951555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:54.951555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:54.951555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:54.951555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:54.951555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:54.951555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:54.951555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:54.951555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:54.951555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:54.951555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:54.951555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:54.951555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:54.951555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:54.951555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:54.951555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:54.952443 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:19:54.952443 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:19:54.952443 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:19:54.952443 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:19:54.952443 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:19:54.952443 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:19:54.952443 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:19:54.952443 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:54.954538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 279/569] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:19:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:54.955561 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da41d26a-266e-44a6-b324-1528e8b4f77a None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:54.956275 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5966d8-9c54-419d-98db-32558366fc76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:54.961860 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-3b5966d8-9c54-419d-98db-32558366fc76 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:19:54.961860 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:54.961860 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:54.961860 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:54.961860 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:54.961860 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:54.961860 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:54.961860 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:54.961860 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:54.961860 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:54.961860 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:54.961860 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:54.961860 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:54.961860 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:54.961860 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:54.961860 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:54.962847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:54.962847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:54.962847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:54.962847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:54.962847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:54.962847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:54.962847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:54.962847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:54.962847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:54.962847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:54.962847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:54.962847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:54.962847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:54.962847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:54.962847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:54.962847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:54.963669 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:54.963669 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:54.963669 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:54.963669 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:54.963669 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:54.963669 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:54.963669 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:54.963669 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:54.963669 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:54.963669 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:54.963669 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:19:54.963669 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:54.963669 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 291/570] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 10:19:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:54.966381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5966d8-9c54-419d-98db-32558366fc76 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:55.016770 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:55.041230 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:55.059711 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da41d26a-266e-44a6-b324-1528e8b4f77a None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:55.073411 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5966d8-9c54-419d-98db-32558366fc76 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:55.093719 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9364ba-7b7b-4d80-8df9-bcd692987371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:55.094375 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9364ba-7b7b-4d80-8df9-bcd692987371 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:55.094652 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9364ba-7b7b-4d80-8df9-bcd692987371 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:55.094923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9364ba-7b7b-4d80-8df9-bcd692987371 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:55.101460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9364ba-7b7b-4d80-8df9-bcd692987371 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:55.102913 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9364ba-7b7b-4d80-8df9-bcd692987371 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:55.107283 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694b6ea-c8fc-49eb-83e4-7ca37f57fc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:55.107884 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694b6ea-c8fc-49eb-83e4-7ca37f57fc64 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:55.108162 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694b6ea-c8fc-49eb-83e4-7ca37f57fc64 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:55.108432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694b6ea-c8fc-49eb-83e4-7ca37f57fc64 None admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:55.109113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3694b6ea-c8fc-49eb-83e4-7ca37f57fc64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:55.109416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3694b6ea-c8fc-49eb-83e4-7ca37f57fc64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:55.110837 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3694b6ea-c8fc-49eb-83e4-7ca37f57fc64 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:55.113926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-bf9364ba-7b7b-4d80-8df9-bcd692987371 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:19:55.113926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:55.113926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:55.113926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:55.113926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:55.113926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:55.113926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:55.113926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:55.113926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:55.113926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:55.113926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:55.113926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:55.113926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:19:55.113926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:19:55.113926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:19:55.114955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:19:55.114955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:55.114955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:55.114955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:55.114955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:55.114955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:55.114955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:55.114955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:55.114955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:55.114955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:55.114955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:55.114955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:55.114955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:55.114955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:55.114955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:55.114955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:55.114955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:55.115874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:19:55.115874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:19:55.115874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:19:55.115874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:19:55.115874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:19:55.115874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:19:55.115874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:19:55.115874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:55.115874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 280/571] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:19:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:55.116441 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9364ba-7b7b-4d80-8df9-bcd692987371 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:55.123320 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 292/572] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:19:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:55.123999 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694b6ea-c8fc-49eb-83e4-7ca37f57fc64 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:55.219475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9364ba-7b7b-4d80-8df9-bcd692987371 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:55.219684 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694b6ea-c8fc-49eb-83e4-7ca37f57fc64 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:55.221907 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:55.253197 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf363e-ba54-45b5-bd22-657e0965b297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:55.253861 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf363e-ba54-45b5-bd22-657e0965b297 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:55.254013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb2aa3-3d18-4432-a386-04e65e24e374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:55.254168 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf363e-ba54-45b5-bd22-657e0965b297 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:55.254426 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf363e-ba54-45b5-bd22-657e0965b297 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:55.254631 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb2aa3-3d18-4432-a386-04e65e24e374 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:55.254906 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb2aa3-3d18-4432-a386-04e65e24e374 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:55.255177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb2aa3-3d18-4432-a386-04e65e24e374 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:55.255823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bb2aa3-3d18-4432-a386-04e65e24e374 None admin] RBAC: Adding query filter params (name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:55.256156 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bb2aa3-3d18-4432-a386-04e65e24e374 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:55.257502 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bb2aa3-3d18-4432-a386-04e65e24e374 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:55.264535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf363e-ba54-45b5-bd22-657e0965b297 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:55.265453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 293/573] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 10:19:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:55.266249 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb2aa3-3d18-4432-a386-04e65e24e374 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:55.270133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-4acf363e-ba54-45b5-bd22-657e0965b297 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:19:55.270133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:55.270133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:55.270133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:55.270133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:55.270133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:55.270133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:55.270133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:55.270133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:55.270133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:55.270133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:55.270133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:55.270133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:55.270133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:55.270133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:55.270133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:55.271104 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:55.271104 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:55.271104 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:55.271104 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:55.271104 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:55.271104 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:55.271104 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:55.271104 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:55.271104 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:55.271104 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:55.271104 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:55.271104 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:55.271104 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:55.271104 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:55.271104 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:55.271104 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:55.271945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:55.271945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:55.271945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:55.271945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:55.271945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:55.271945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:55.271945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:55.271945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:55.271945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:55.271945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:55.271945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:19:55.271945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:55.271945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 281/574] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 10:19:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:55.271945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf363e-ba54-45b5-bd22-657e0965b297 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:55.274535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:55.368581 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb2aa3-3d18-4432-a386-04e65e24e374 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:55.370828 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf363e-ba54-45b5-bd22-657e0965b297 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:55.400711 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7518f742-b225-408f-b439-826404889317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:55.401329 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518f742-b225-408f-b439-826404889317 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:55.401608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518f742-b225-408f-b439-826404889317 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:55.401877 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518f742-b225-408f-b439-826404889317 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:55.404431 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e4763-4de4-48d3-bb4d-0781ade91d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:55.405055 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e4763-4de4-48d3-bb4d-0781ade91d3a None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:55.405323 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e4763-4de4-48d3-bb4d-0781ade91d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:55.405588 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e4763-4de4-48d3-bb4d-0781ade91d3a None admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:55.406295 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e4763-4de4-48d3-bb4d-0781ade91d3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:55.406592 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e4763-4de4-48d3-bb4d-0781ade91d3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:55.408013 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e4763-4de4-48d3-bb4d-0781ade91d3a None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:55.412727 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7518f742-b225-408f-b439-826404889317 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:55.418504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-7518f742-b225-408f-b439-826404889317 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:19:55.418504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:55.418504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:55.418504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:55.418504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:55.418504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:55.418504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:55.418504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:55.418504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:55.418504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:55.418504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:55.418504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:55.418504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:55.418504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:55.418504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:55.418504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:55.420179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:55.420179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:55.420179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:55.420179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:55.420179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:55.420179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:55.420179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:55.420179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:55.420179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:55.420179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:55.420179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:55.420179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:55.420179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:55.420179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:55.420179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:55.420179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:55.421153 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:55.421153 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:55.421153 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:55.421153 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:55.421153 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:55.421153 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:55.421153 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:55.421153 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:55.421153 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:55.421153 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:55.421153 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:19:55.421153 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:55.421153 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 294/575] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 10:19:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:55.421153 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7518f742-b225-408f-b439-826404889317 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:55.422420 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 282/576] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:19:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:55.423526 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e4763-4de4-48d3-bb4d-0781ade91d3a None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:55.447770 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:55.485792 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:55.520760 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7518f742-b225-408f-b439-826404889317 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:55.523651 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e4763-4de4-48d3-bb4d-0781ade91d3a None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:55.554532 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fbfe0-4b3b-4732-8d76-c5bcdabdf80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:55.555122 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fbfe0-4b3b-4732-8d76-c5bcdabdf80e None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:55.555405 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fbfe0-4b3b-4732-8d76-c5bcdabdf80e None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:55.555676 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fbfe0-4b3b-4732-8d76-c5bcdabdf80e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:55.555966 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2b0a3-a3a4-4409-9183-74bf47701b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:55.556386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28fbfe0-4b3b-4732-8d76-c5bcdabdf80e None admin] RBAC: Adding query filter params (role.id=d36e58344b604bcda1cee49065e8497e, scope.project.id=cff3a6011f4c4893bddffdad3d69f3df, user.id=5e6be34fc93a4551a83e5b8380188a6b) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:55.556579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2b0a3-a3a4-4409-9183-74bf47701b2d None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:55.556679 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28fbfe0-4b3b-4732-8d76-c5bcdabdf80e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:55.556853 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2b0a3-a3a4-4409-9183-74bf47701b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:55.557120 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2b0a3-a3a4-4409-9183-74bf47701b2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:55.558141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28fbfe0-4b3b-4732-8d76-c5bcdabdf80e None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:55.564018 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 295/577] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:19:55 2022] GET /identity/v3/role_assignments?user.id=5e6be34fc93a4551a83e5b8380188a6b&scope.project.id=cff3a6011f4c4893bddffdad3d69f3df&role.id=d36e58344b604bcda1cee49065e8497e => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:55.564792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fbfe0-4b3b-4732-8d76-c5bcdabdf80e None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:55.568995 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2b0a3-a3a4-4409-9183-74bf47701b2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:55.570509 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2b0a3-a3a4-4409-9183-74bf47701b2d None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:55.587498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-50f2b0a3-a3a4-4409-9183-74bf47701b2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:55.587498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:55.587498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:55.587498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:55.587498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:55.587498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:55.587498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:55.587498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:55.587498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:55.587498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:55.587498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:55.587498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:55.587498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:19:55.587498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:19:55.587498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:19:55.587498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:19:55.588517 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:55.588517 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:55.588517 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:55.588517 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:55.588517 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:55.588517 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:55.588517 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:55.588517 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:55.588517 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:55.588517 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:55.588517 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:55.588517 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:55.588517 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:55.588517 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:55.588517 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:55.588517 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:55.588517 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:19:55.589397 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:19:55.589397 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:55.589397 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:55.589397 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 283/578] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:55.589632 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2b0a3-a3a4-4409-9183-74bf47701b2d None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:55.679069 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:55.690521 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fbfe0-4b3b-4732-8d76-c5bcdabdf80e None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:55.708165 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2b0a3-a3a4-4409-9183-74bf47701b2d None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:55.724700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:55.728200 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8a58a-f29c-4cb7-886a-891d2f423ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:55.728936 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8a58a-f29c-4cb7-886a-891d2f423ebe None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:55.729117 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8a58a-f29c-4cb7-886a-891d2f423ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:55.729354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8a58a-f29c-4cb7-886a-891d2f423ebe None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:55.730053 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad8a58a-f29c-4cb7-886a-891d2f423ebe None admin] RBAC: Adding query filter params (name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:55.730339 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad8a58a-f29c-4cb7-886a-891d2f423ebe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:55.732247 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad8a58a-f29c-4cb7-886a-891d2f423ebe None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:55.740963 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 296/579] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 10:19:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:55.745770 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8a58a-f29c-4cb7-886a-891d2f423ebe None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:55.755760 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a287c-97f7-4dd9-a19a-65ccae4b7f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:55.756370 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a287c-97f7-4dd9-a19a-65ccae4b7f0f None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:55.756833 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a287c-97f7-4dd9-a19a-65ccae4b7f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:55.757838 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a287c-97f7-4dd9-a19a-65ccae4b7f0f None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:55.758560 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a287c-97f7-4dd9-a19a-65ccae4b7f0f None admin] RBAC: Adding query filter params (name=service) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:55.758863 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a287c-97f7-4dd9-a19a-65ccae4b7f0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:55.760590 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a287c-97f7-4dd9-a19a-65ccae4b7f0f None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:55.769247 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 284/580] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 10:19:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:55.770183 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a287c-97f7-4dd9-a19a-65ccae4b7f0f None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:55.864757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8a58a-f29c-4cb7-886a-891d2f423ebe None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:55.880480 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a287c-97f7-4dd9-a19a-65ccae4b7f0f None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:55.905092 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e8786-481e-441c-b10b-dc056ca3c045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:55.905744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e8786-481e-441c-b10b-dc056ca3c045 None admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:55.906015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e8786-481e-441c-b10b-dc056ca3c045 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:55.906301 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e8786-481e-441c-b10b-dc056ca3c045 None admin] PATH_INFO: `/v3/projects/cff3a6011f4c4893bddffdad3d69f3df/users/cee41014d5d043888c6b399710bccab4/roles/d36e58344b604bcda1cee49065e8497e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:55.915674 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79cd67e-af19-4681-aebf-d24a9ebc1e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:55.916282 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cd67e-af19-4681-aebf-d24a9ebc1e64 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:55.916561 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cd67e-af19-4681-aebf-d24a9ebc1e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:55.916833 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cd67e-af19-4681-aebf-d24a9ebc1e64 None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:55.917473 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79cd67e-af19-4681-aebf-d24a9ebc1e64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:55.917775 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79cd67e-af19-4681-aebf-d24a9ebc1e64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:55.919241 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79cd67e-af19-4681-aebf-d24a9ebc1e64 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:55.922772 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e8786-481e-441c-b10b-dc056ca3c045 None admin] RBAC: Authorizing `identity:create_grant(project_id=cff3a6011f4c4893bddffdad3d69f3df, user_id=cee41014d5d043888c6b399710bccab4, role_id=d36e58344b604bcda1cee49065e8497e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:55.924174 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e8786-481e-441c-b10b-dc056ca3c045 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:55.934070 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 285/581] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:19:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:55.935273 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79cd67e-af19-4681-aebf-d24a9ebc1e64 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:55.936609 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 297/582] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 10:19:55 2022] PUT /identity/v3/projects/cff3a6011f4c4893bddffdad3d69f3df/users/cee41014d5d043888c6b399710bccab4/roles/d36e58344b604bcda1cee49065e8497e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:19:55.937410 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e8786-481e-441c-b10b-dc056ca3c045 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:55.942344 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:55.988216 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:56.064301 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79cd67e-af19-4681-aebf-d24a9ebc1e64 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:56.080282 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e8786-481e-441c-b10b-dc056ca3c045 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:56.102990 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121aaf49-d960-446e-8cf9-cd4c9c73266a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:56.103845 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121aaf49-d960-446e-8cf9-cd4c9c73266a None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:56.103980 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121aaf49-d960-446e-8cf9-cd4c9c73266a None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:56.104148 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121aaf49-d960-446e-8cf9-cd4c9c73266a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:56.116147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121aaf49-d960-446e-8cf9-cd4c9c73266a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:56.117695 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121aaf49-d960-446e-8cf9-cd4c9c73266a None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:56.125852 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332b0c3-e3f3-48bf-acb8-7363e5ce792b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:56.126761 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332b0c3-e3f3-48bf-acb8-7363e5ce792b None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:56.127045 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332b0c3-e3f3-48bf-acb8-7363e5ce792b None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:56.127319 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332b0c3-e3f3-48bf-acb8-7363e5ce792b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:56.134480 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-121aaf49-d960-446e-8cf9-cd4c9c73266a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:56.134480 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:56.134480 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:56.134480 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:56.134480 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:56.134480 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:56.134480 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:56.134480 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:56.134480 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:56.134480 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:56.134480 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:56.134480 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:56.134480 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:19:56.134480 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:19:56.134480 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:19:56.134480 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:19:56.135412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:56.135412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:56.135412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:56.135412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:56.135412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:56.135412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:56.135412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:56.135412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:56.135412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:56.135412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:56.135412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:56.135412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:56.135412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:56.135412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:56.135412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:56.135412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:56.135412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:19:56.136277 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:19:56.136277 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:56.136277 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:56.136277 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 286/583] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:56.136897 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121aaf49-d960-446e-8cf9-cd4c9c73266a None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:56.139002 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3332b0c3-e3f3-48bf-acb8-7363e5ce792b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:56.141472 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3332b0c3-e3f3-48bf-acb8-7363e5ce792b None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:56.157099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-3332b0c3-e3f3-48bf-acb8-7363e5ce792b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:56.157099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:56.157099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:56.157099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:56.157099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:56.157099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:56.157099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:56.157099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:56.157099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:56.157099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:56.157099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:56.157099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:56.157099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:19:56.157099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:19:56.157099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:19:56.157099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:19:56.158066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:56.158066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:56.158066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:56.158066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:56.158066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:56.158066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:56.158066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:56.158066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:56.158066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:56.158066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:56.158066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:56.158066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:56.158066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:56.158066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:56.158066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:56.158066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:56.158066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:19:56.158965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:19:56.158965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:56.158965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:56.158965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 298/584] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:56.159343 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332b0c3-e3f3-48bf-acb8-7363e5ce792b None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:56.234918 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:56.243841 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:56.257380 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121aaf49-d960-446e-8cf9-cd4c9c73266a None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:56.292458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ebd12-2109-45ff-91a0-70a263f7e967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:56.292922 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332b0c3-e3f3-48bf-acb8-7363e5ce792b None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:56.293268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ebd12-2109-45ff-91a0-70a263f7e967 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:56.293611 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ebd12-2109-45ff-91a0-70a263f7e967 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:56.293948 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ebd12-2109-45ff-91a0-70a263f7e967 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:56.302987 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8ebd12-2109-45ff-91a0-70a263f7e967 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:56.304527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8ebd12-2109-45ff-91a0-70a263f7e967 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:56.316690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-2a8ebd12-2109-45ff-91a0-70a263f7e967 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 26 10:19:56.316690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:56.316690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:56.316690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:56.316690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:56.316690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:56.316690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:56.316690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:56.316690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:56.316690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:56.316690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:56.316690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:56.316690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:19:56.316690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:19:56.316690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:19:56.316690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:19:56.317700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:56.317700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:56.317700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:19:56.317700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:19:56.317700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:19:56.317700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:19:56.317700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:56.317700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:56.317700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:56.317700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:56.317700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:56.317700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:56.317700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:56.317700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:56.317700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:56.317700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:56.317700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:56.317700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:56.318658 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:56.318658 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:56.318658 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:19:56.318658 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:19:56.318658 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:19:56.318658 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:19:56.318658 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:19:56.318658 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:19:56.318658 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 26 10:19:56.318658 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:56.318658 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 287/585] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 10:19:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:56.323710 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ebd12-2109-45ff-91a0-70a263f7e967 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:56.332283 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caecf74a-70ae-45e4-afbd-96b507eb993f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:56.332895 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecf74a-70ae-45e4-afbd-96b507eb993f None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:56.333174 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecf74a-70ae-45e4-afbd-96b507eb993f None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:56.333445 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecf74a-70ae-45e4-afbd-96b507eb993f None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:56.334111 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caecf74a-70ae-45e4-afbd-96b507eb993f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:56.334417 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caecf74a-70ae-45e4-afbd-96b507eb993f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:56.335925 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caecf74a-70ae-45e4-afbd-96b507eb993f None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:56.350389 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 299/586] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:19:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:56.351549 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caecf74a-70ae-45e4-afbd-96b507eb993f None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:56.432935 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ebd12-2109-45ff-91a0-70a263f7e967 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:56.470744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e1dcd-9471-4d05-90cc-4eaf0bf0c751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:56.471588 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e1dcd-9471-4d05-90cc-4eaf0bf0c751 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:56.472024 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e1dcd-9471-4d05-90cc-4eaf0bf0c751 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:56.472386 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e1dcd-9471-4d05-90cc-4eaf0bf0c751 None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:56.473075 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872e1dcd-9471-4d05-90cc-4eaf0bf0c751 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:56.473381 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872e1dcd-9471-4d05-90cc-4eaf0bf0c751 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:56.474766 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872e1dcd-9471-4d05-90cc-4eaf0bf0c751 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:56.478591 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caecf74a-70ae-45e4-afbd-96b507eb993f None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:56.487552 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 288/587] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:19:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:56.488355 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e1dcd-9471-4d05-90cc-4eaf0bf0c751 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:56.495926 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:56.500717 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:56.521656 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d1c317-fed9-425d-b95c-4c5be03f6ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:56.522296 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1c317-fed9-425d-b95c-4c5be03f6ade None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:56.522575 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1c317-fed9-425d-b95c-4c5be03f6ade None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:56.523094 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1c317-fed9-425d-b95c-4c5be03f6ade None admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:56.524228 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d1c317-fed9-425d-b95c-4c5be03f6ade None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:56.524661 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d1c317-fed9-425d-b95c-4c5be03f6ade None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:56.526235 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d1c317-fed9-425d-b95c-4c5be03f6ade None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:56.545977 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 300/588] 10.222.0.49 () {56 vars in 1139 bytes} [Mon Sep 26 10:19:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:56.546831 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d1c317-fed9-425d-b95c-4c5be03f6ade None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:56.611787 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e1dcd-9471-4d05-90cc-4eaf0bf0c751 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:56.655980 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f552f7c8-0a38-4425-a2b9-cc78d39fad5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:56.657063 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552f7c8-0a38-4425-a2b9-cc78d39fad5e None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:56.657461 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552f7c8-0a38-4425-a2b9-cc78d39fad5e None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:56.657805 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552f7c8-0a38-4425-a2b9-cc78d39fad5e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:56.666087 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d1c317-fed9-425d-b95c-4c5be03f6ade None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:56.666750 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f552f7c8-0a38-4425-a2b9-cc78d39fad5e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:56.668390 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f552f7c8-0a38-4425-a2b9-cc78d39fad5e None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:56.684718 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-f552f7c8-0a38-4425-a2b9-cc78d39fad5e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 10:19:56.684718 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:56.684718 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:56.684718 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:56.684718 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:56.684718 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:56.684718 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:56.684718 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:56.684718 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:56.684718 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:56.684718 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:56.684718 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:56.684718 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:19:56.684718 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:19:56.684718 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:19:56.684718 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:19:56.687174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:56.687174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:56.687174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:19:56.687174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:19:56.687174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:19:56.687174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:19:56.687174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:56.687174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:56.687174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:56.687174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:56.687174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:56.687174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:56.687174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:56.687174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:56.687174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:56.687174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:56.687174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:56.687174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:56.688617 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:56.688617 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:56.688617 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:19:56.688617 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:19:56.688617 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:19:56.688617 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:19:56.688617 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:19:56.688617 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:19:56.688617 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 10:19:56.688617 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:56.688617 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 289/589] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:19:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:56.688617 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f552f7c8-0a38-4425-a2b9-cc78d39fad5e None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:56.708798 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f34b9-5159-4f29-8fd8-d2329aa4b234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:56.709731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f34b9-5159-4f29-8fd8-d2329aa4b234 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:56.710111 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f34b9-5159-4f29-8fd8-d2329aa4b234 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:56.710576 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f34b9-5159-4f29-8fd8-d2329aa4b234 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:56.720036 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f34b9-5159-4f29-8fd8-d2329aa4b234 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:56.721753 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f34b9-5159-4f29-8fd8-d2329aa4b234 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:56.735999 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-fc3f34b9-5159-4f29-8fd8-d2329aa4b234 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 26 10:19:56.735999 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:56.735999 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:56.735999 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:56.735999 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:56.735999 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:56.735999 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:56.735999 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:56.735999 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:56.735999 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:56.735999 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:56.735999 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:56.735999 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:19:56.735999 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:19:56.735999 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:19:56.735999 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:19:56.737889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:56.737889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:56.737889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:19:56.737889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:19:56.737889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:19:56.737889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:19:56.737889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:56.737889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:56.737889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:56.737889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:56.737889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:56.737889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:56.737889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:56.737889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:56.737889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:56.737889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:56.737889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:56.737889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:56.740114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:56.740114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:56.740114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:19:56.740114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:19:56.740114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:19:56.740114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:19:56.740114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:19:56.740114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:19:56.740114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 26 10:19:56.740114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:56.740114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 301/590] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:19:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:56.740114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f34b9-5159-4f29-8fd8-d2329aa4b234 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:56.770099 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:56.778446 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:56.820960 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f552f7c8-0a38-4425-a2b9-cc78d39fad5e None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:56.862153 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b848e300-6edd-466b-baa9-a5d57736ba6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:56.863020 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848e300-6edd-466b-baa9-a5d57736ba6a None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:56.863455 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848e300-6edd-466b-baa9-a5d57736ba6a None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:56.863913 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848e300-6edd-466b-baa9-a5d57736ba6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:56.865714 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f34b9-5159-4f29-8fd8-d2329aa4b234 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:56.875619 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b848e300-6edd-466b-baa9-a5d57736ba6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:56.877357 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b848e300-6edd-466b-baa9-a5d57736ba6a None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:56.891816 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-b848e300-6edd-466b-baa9-a5d57736ba6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:56.891816 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:56.891816 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:56.891816 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:56.891816 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:56.891816 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:56.891816 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:56.891816 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:56.891816 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:56.891816 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:56.891816 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:56.891816 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:56.891816 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:19:56.891816 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:19:56.891816 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:19:56.891816 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:19:56.892804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:56.892804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:56.892804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:56.892804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:56.892804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:56.892804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:56.892804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:56.892804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:56.892804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:56.892804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:56.892804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:56.892804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:56.892804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:56.892804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:56.892804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:56.892804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:56.892804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:19:56.893706 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:19:56.893706 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:56.893706 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:56.900501 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 290/591] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:56.901816 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b848e300-6edd-466b-baa9-a5d57736ba6a None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:56.904837 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306eaf87-f1f8-4b60-9e2f-f55ee09598dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:56.905578 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306eaf87-f1f8-4b60-9e2f-f55ee09598dd None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:56.905923 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306eaf87-f1f8-4b60-9e2f-f55ee09598dd None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:56.906293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306eaf87-f1f8-4b60-9e2f-f55ee09598dd None admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:56.907116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306eaf87-f1f8-4b60-9e2f-f55ee09598dd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:56.907490 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306eaf87-f1f8-4b60-9e2f-f55ee09598dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:56.909016 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306eaf87-f1f8-4b60-9e2f-f55ee09598dd None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:56.928315 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 302/592] 10.222.0.49 () {56 vars in 1137 bytes} [Mon Sep 26 10:19:56 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:56.929332 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306eaf87-f1f8-4b60-9e2f-f55ee09598dd None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:57.008364 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b848e300-6edd-466b-baa9-a5d57736ba6a None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:57.024470 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:57.029729 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:57.035876 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306eaf87-f1f8-4b60-9e2f-f55ee09598dd None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:57.045541 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2584f9-abfb-4b90-9452-4bc6850047d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:57.046434 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2584f9-abfb-4b90-9452-4bc6850047d2 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:57.046926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2584f9-abfb-4b90-9452-4bc6850047d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:57.047401 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2584f9-abfb-4b90-9452-4bc6850047d2 None admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:57.048313 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2584f9-abfb-4b90-9452-4bc6850047d2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:57.048823 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2584f9-abfb-4b90-9452-4bc6850047d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:57.050472 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2584f9-abfb-4b90-9452-4bc6850047d2 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:57.068817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 291/593] 10.222.0.49 () {56 vars in 1137 bytes} [Mon Sep 26 10:19:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:57.070639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2584f9-abfb-4b90-9452-4bc6850047d2 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:57.074178 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41978a-b154-43b3-9154-1cbe94813d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:57.075279 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41978a-b154-43b3-9154-1cbe94813d68 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:57.075279 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41978a-b154-43b3-9154-1cbe94813d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:57.075523 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41978a-b154-43b3-9154-1cbe94813d68 None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:57.076102 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f41978a-b154-43b3-9154-1cbe94813d68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:57.076403 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f41978a-b154-43b3-9154-1cbe94813d68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:57.077824 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f41978a-b154-43b3-9154-1cbe94813d68 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:57.094609 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 303/594] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:19:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:57.094609 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41978a-b154-43b3-9154-1cbe94813d68 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:57.194312 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2584f9-abfb-4b90-9452-4bc6850047d2 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:57.218481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41978a-b154-43b3-9154-1cbe94813d68 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:57.240989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d6e33e-b045-4881-be6f-42a5c7f2a1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:57.240989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6e33e-b045-4881-be6f-42a5c7f2a1de None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:57.240989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6e33e-b045-4881-be6f-42a5c7f2a1de None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:57.240989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6e33e-b045-4881-be6f-42a5c7f2a1de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:57.252561 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d6e33e-b045-4881-be6f-42a5c7f2a1de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:57.254938 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d6e33e-b045-4881-be6f-42a5c7f2a1de None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:57.258245 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948535a0-eb7e-41de-b772-e14ec0148874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:57.258947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948535a0-eb7e-41de-b772-e14ec0148874 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:57.259289 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948535a0-eb7e-41de-b772-e14ec0148874 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:57.259625 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948535a0-eb7e-41de-b772-e14ec0148874 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:57.272375 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948535a0-eb7e-41de-b772-e14ec0148874 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:57.272956 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-24d6e33e-b045-4881-be6f-42a5c7f2a1de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:57.272956 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:57.272956 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:57.272956 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:57.272956 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:57.272956 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:57.272956 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:57.272956 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:57.272956 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:57.272956 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:57.272956 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:57.272956 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:57.272956 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:19:57.272956 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:19:57.272956 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:19:57.272956 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:19:57.274093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:57.274093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:57.274093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:57.274093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:57.274093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:57.274093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:57.274093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:57.274093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:57.274093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:57.274093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:57.274093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:57.274093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:57.274093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:57.274093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:57.274093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:57.274093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:57.274093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:19:57.275211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:19:57.275211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:57.275211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:57.280767 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948535a0-eb7e-41de-b772-e14ec0148874 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:57.281230 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 292/595] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:57.282237 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d6e33e-b045-4881-be6f-42a5c7f2a1de None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:57.294895 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-948535a0-eb7e-41de-b772-e14ec0148874 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:57.294895 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:57.294895 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:57.294895 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:57.294895 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:57.294895 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:57.294895 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:57.294895 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:57.294895 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:57.294895 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:57.294895 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:57.294895 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:57.294895 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:19:57.294895 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:19:57.294895 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:19:57.294895 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:19:57.297148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:57.297148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:57.297148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:57.297148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:57.297148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:57.297148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:57.297148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:57.297148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:57.297148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:57.297148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:57.297148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:57.297148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:57.297148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:57.297148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:57.297148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:57.297148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:57.297148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:19:57.298623 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:19:57.298623 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:57.298623 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:57.298623 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 304/596] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:57.298623 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948535a0-eb7e-41de-b772-e14ec0148874 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:57.303948 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:57.329885 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:57.412414 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948535a0-eb7e-41de-b772-e14ec0148874 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:57.422677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d6e33e-b045-4881-be6f-42a5c7f2a1de None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:57.447814 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81107f6-7bdf-4cb0-80be-2693a7d705eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:57.448297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81107f6-7bdf-4cb0-80be-2693a7d705eb None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:57.448667 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81107f6-7bdf-4cb0-80be-2693a7d705eb None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:57.449029 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81107f6-7bdf-4cb0-80be-2693a7d705eb None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:57.449765 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81107f6-7bdf-4cb0-80be-2693a7d705eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:57.450196 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81107f6-7bdf-4cb0-80be-2693a7d705eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:57.451762 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81107f6-7bdf-4cb0-80be-2693a7d705eb None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:57.460003 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b53c0-d704-4f13-8bac-b98153ce65f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:57.460815 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b53c0-d704-4f13-8bac-b98153ce65f7 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:57.461237 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b53c0-d704-4f13-8bac-b98153ce65f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:57.461831 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b53c0-d704-4f13-8bac-b98153ce65f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:57.465366 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 305/597] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:57.466943 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81107f6-7bdf-4cb0-80be-2693a7d705eb None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:57.470170 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740b53c0-d704-4f13-8bac-b98153ce65f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:57.471937 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740b53c0-d704-4f13-8bac-b98153ce65f7 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:57.484914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-740b53c0-d704-4f13-8bac-b98153ce65f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:19:57.484914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:57.484914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:57.484914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:57.484914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:57.484914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:57.484914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:57.484914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:57.484914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:57.484914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:57.484914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:57.484914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:57.484914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:19:57.484914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:19:57.484914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:19:57.485900 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:19:57.485900 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:57.485900 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:57.485900 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:57.485900 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:57.485900 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:57.485900 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:57.485900 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:57.485900 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:57.485900 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:57.485900 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:57.485900 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:57.485900 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:57.485900 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:57.485900 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:57.485900 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:57.485900 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:57.486818 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:19:57.486818 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:19:57.486818 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:19:57.486818 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:19:57.486818 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:19:57.486818 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:19:57.486818 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:19:57.486818 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:57.493610 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 293/598] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:19:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:57.494796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b53c0-d704-4f13-8bac-b98153ce65f7 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:57.530861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:57.570832 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81107f6-7bdf-4cb0-80be-2693a7d705eb None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:57.598442 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:57.604537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b53c0-d704-4f13-8bac-b98153ce65f7 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:57.608210 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2c4d93-d020-4e02-837e-d04a65602331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:57.608210 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c4d93-d020-4e02-837e-d04a65602331 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:57.608459 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c4d93-d020-4e02-837e-d04a65602331 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:57.608633 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c4d93-d020-4e02-837e-d04a65602331 None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:57.609389 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2c4d93-d020-4e02-837e-d04a65602331 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:57.609753 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2c4d93-d020-4e02-837e-d04a65602331 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:57.611270 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2c4d93-d020-4e02-837e-d04a65602331 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:57.624810 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 306/599] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:57.625899 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2c4d93-d020-4e02-837e-d04a65602331 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:57.646790 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0833c3-02dc-4532-97b7-b7a4a0c1e971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:57.647914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0833c3-02dc-4532-97b7-b7a4a0c1e971 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:57.648415 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0833c3-02dc-4532-97b7-b7a4a0c1e971 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:57.648877 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0833c3-02dc-4532-97b7-b7a4a0c1e971 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:57.657540 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0833c3-02dc-4532-97b7-b7a4a0c1e971 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:57.659480 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0833c3-02dc-4532-97b7-b7a4a0c1e971 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:57.672527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-cf0833c3-02dc-4532-97b7-b7a4a0c1e971 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:19:57.672527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:57.672527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:57.672527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:57.672527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:57.672527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:57.672527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:57.672527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:57.672527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:57.672527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:57.672527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:57.672527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:57.672527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:19:57.672527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:19:57.672527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:19:57.673421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:19:57.673421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:57.673421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:57.673421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:57.673421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:57.673421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:57.673421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:57.673421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:57.673421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:57.673421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:57.673421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:57.673421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:57.673421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:57.673421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:57.673421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:57.673421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:57.673421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:57.674325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:19:57.674325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:19:57.674325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:19:57.674325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:19:57.674325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:19:57.674325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:19:57.674325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:19:57.674325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:57.676652 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 294/600] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:19:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:57.677717 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0833c3-02dc-4532-97b7-b7a4a0c1e971 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:57.735137 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2c4d93-d020-4e02-837e-d04a65602331 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:57.758403 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:57.773352 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06434cef-ee04-40b9-b54a-a1fe66916e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:57.773961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06434cef-ee04-40b9-b54a-a1fe66916e6e None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:57.774265 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06434cef-ee04-40b9-b54a-a1fe66916e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:57.774537 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06434cef-ee04-40b9-b54a-a1fe66916e6e None admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:57.775228 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06434cef-ee04-40b9-b54a-a1fe66916e6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:57.775530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06434cef-ee04-40b9-b54a-a1fe66916e6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:57.776979 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06434cef-ee04-40b9-b54a-a1fe66916e6e None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:57.789148 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0833c3-02dc-4532-97b7-b7a4a0c1e971 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:57.795229 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 307/601] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:19:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:57.796128 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06434cef-ee04-40b9-b54a-a1fe66916e6e None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:57.828524 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8bd4f-d1b5-485d-92c0-caccacab6626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:57.829244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8bd4f-d1b5-485d-92c0-caccacab6626 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:57.829530 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8bd4f-d1b5-485d-92c0-caccacab6626 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:57.829804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8bd4f-d1b5-485d-92c0-caccacab6626 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:57.836955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8bd4f-d1b5-485d-92c0-caccacab6626 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:57.838406 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8bd4f-d1b5-485d-92c0-caccacab6626 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:57.850800 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-92c8bd4f-d1b5-485d-92c0-caccacab6626 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:19:57.850800 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:57.850800 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:57.850800 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:57.850800 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:57.850800 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:57.850800 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:57.850800 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:57.850800 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:57.850800 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:57.850800 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:57.850800 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:57.850800 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:19:57.850800 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:19:57.850800 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:19:57.851703 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:19:57.851703 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:57.851703 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:57.851703 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:57.851703 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:57.851703 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:57.851703 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:57.851703 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:57.851703 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:57.851703 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:57.851703 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:57.851703 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:57.851703 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:57.851703 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:57.851703 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:57.851703 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:57.851703 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:57.852601 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:19:57.852601 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:19:57.852601 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:19:57.852601 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:19:57.852601 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:19:57.852601 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:19:57.852601 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:19:57.852601 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:57.852601 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 295/602] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:19:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:57.853084 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3fe88-150b-4254-b796-b3934d826bab None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:57.853462 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3fe88-150b-4254-b796-b3934d826bab None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:57.853633 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3fe88-150b-4254-b796-b3934d826bab None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:57.859386 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 296/603] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:57.860233 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3fe88-150b-4254-b796-b3934d826bab None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:57.873656 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:57.911146 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06434cef-ee04-40b9-b54a-a1fe66916e6e None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:57.947307 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a57e1-dd6e-4129-9c85-7611674b9be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:57.947932 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a57e1-dd6e-4129-9c85-7611674b9be0 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:57.948304 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a57e1-dd6e-4129-9c85-7611674b9be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:57.948579 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a57e1-dd6e-4129-9c85-7611674b9be0 None admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:57.949268 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2a57e1-dd6e-4129-9c85-7611674b9be0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:57.949579 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2a57e1-dd6e-4129-9c85-7611674b9be0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:57.951655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2a57e1-dd6e-4129-9c85-7611674b9be0 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:57.965139 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 308/604] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:19:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:57.966778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a57e1-dd6e-4129-9c85-7611674b9be0 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:57.975335 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3fe88-150b-4254-b796-b3934d826bab None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:58.011884 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:58.014700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7539ec-46b0-4e47-89db-3969216731c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:58.015604 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7539ec-46b0-4e47-89db-3969216731c4 None admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:58.016070 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7539ec-46b0-4e47-89db-3969216731c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:58.016342 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7539ec-46b0-4e47-89db-3969216731c4 None admin] PATH_INFO: `/v3/projects/cff3a6011f4c4893bddffdad3d69f3df/users/f841bf31c43b4333a3d834d38b5197e1/roles/d36e58344b604bcda1cee49065e8497e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:58.031670 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7539ec-46b0-4e47-89db-3969216731c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=cff3a6011f4c4893bddffdad3d69f3df, user_id=f841bf31c43b4333a3d834d38b5197e1, role_id=d36e58344b604bcda1cee49065e8497e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:58.033139 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7539ec-46b0-4e47-89db-3969216731c4 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:58.046749 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 297/605] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 10:19:57 2022] PUT /identity/v3/projects/cff3a6011f4c4893bddffdad3d69f3df/users/f841bf31c43b4333a3d834d38b5197e1/roles/d36e58344b604bcda1cee49065e8497e => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:19:58.047930 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2141581-2215-4615-a2ac-6deefe949868 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:58.048208 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2141581-2215-4615-a2ac-6deefe949868 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:58.048474 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2141581-2215-4615-a2ac-6deefe949868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:58.072706 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-b2141581-2215-4615-a2ac-6deefe949868 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:58.074354 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-b2141581-2215-4615-a2ac-6deefe949868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:58.089674 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a57e1-dd6e-4129-9c85-7611674b9be0 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:58.137158 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543ba2f-8f6e-4367-bda6-fda023cd44a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:58.137795 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543ba2f-8f6e-4367-bda6-fda023cd44a4 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:58.138659 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543ba2f-8f6e-4367-bda6-fda023cd44a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:58.138934 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543ba2f-8f6e-4367-bda6-fda023cd44a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:58.139660 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5543ba2f-8f6e-4367-bda6-fda023cd44a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:58.139963 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5543ba2f-8f6e-4367-bda6-fda023cd44a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:58.142543 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5543ba2f-8f6e-4367-bda6-fda023cd44a4 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:58.153722 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:58.160072 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 309/606] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:19:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:58.161756 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543ba2f-8f6e-4367-bda6-fda023cd44a4 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:58.175130 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 298/607] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:19:58.176487 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2141581-2215-4615-a2ac-6deefe949868 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:58.281826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543ba2f-8f6e-4367-bda6-fda023cd44a4 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:58.283482 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:58.323171 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0a46d-5338-477e-90a5-872ddc202c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:58.323830 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0a46d-5338-477e-90a5-872ddc202c0f None admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:58.324108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0a46d-5338-477e-90a5-872ddc202c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:58.324382 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0a46d-5338-477e-90a5-872ddc202c0f None admin] PATH_INFO: `/v3/projects/cff3a6011f4c4893bddffdad3d69f3df/users/feddae5ba018426697854931a55104e3/roles/d36e58344b604bcda1cee49065e8497e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:58.332594 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2141581-2215-4615-a2ac-6deefe949868 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:58.342314 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0a46d-5338-477e-90a5-872ddc202c0f None admin] RBAC: Authorizing `identity:create_grant(project_id=cff3a6011f4c4893bddffdad3d69f3df, user_id=feddae5ba018426697854931a55104e3, role_id=d36e58344b604bcda1cee49065e8497e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:58.344151 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0a46d-5338-477e-90a5-872ddc202c0f None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:58.368317 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 310/608] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 10:19:58 2022] PUT /identity/v3/projects/cff3a6011f4c4893bddffdad3d69f3df/users/feddae5ba018426697854931a55104e3/roles/d36e58344b604bcda1cee49065e8497e => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:19:58.368317 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c196f7-5d3e-48ac-9c08-13547937fb72 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:58.368317 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c196f7-5d3e-48ac-9c08-13547937fb72 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:58.368317 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c196f7-5d3e-48ac-9c08-13547937fb72 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:58.368317 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 311/609] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:19:58.374234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110567b1-5b90-4705-bcd9-a5feaa10f81d None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:58.374517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110567b1-5b90-4705-bcd9-a5feaa10f81d None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:58.374782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110567b1-5b90-4705-bcd9-a5feaa10f81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:58.394295 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-110567b1-5b90-4705-bcd9-a5feaa10f81d None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:19:58.395499 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-110567b1-5b90-4705-bcd9-a5feaa10f81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:19:58.402841 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461dae09-accd-44a6-996f-6cf5148a07ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:58.404427 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461dae09-accd-44a6-996f-6cf5148a07ec None admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:58.404914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461dae09-accd-44a6-996f-6cf5148a07ec None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:58.405403 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461dae09-accd-44a6-996f-6cf5148a07ec None admin] PATH_INFO: `/v3/projects/cff3a6011f4c4893bddffdad3d69f3df/users/41b242fbe2504f8cac26ce4ff46f5390/roles/d36e58344b604bcda1cee49065e8497e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:58.438211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461dae09-accd-44a6-996f-6cf5148a07ec None admin] RBAC: Authorizing `identity:create_grant(project_id=cff3a6011f4c4893bddffdad3d69f3df, user_id=41b242fbe2504f8cac26ce4ff46f5390, role_id=d36e58344b604bcda1cee49065e8497e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:58.439788 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461dae09-accd-44a6-996f-6cf5148a07ec None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:58.472629 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 299/610] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 10:19:58 2022] PUT /identity/v3/projects/cff3a6011f4c4893bddffdad3d69f3df/users/41b242fbe2504f8cac26ce4ff46f5390/roles/d36e58344b604bcda1cee49065e8497e => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:19:58.486388 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 312/611] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:19:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 10:19:58.591145 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461dae09-accd-44a6-996f-6cf5148a07ec None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:58.641718 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:58.710986 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461dae09-accd-44a6-996f-6cf5148a07ec None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:58.751108 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cda40d-29fc-467e-a16e-391bcefff510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:58.752042 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cda40d-29fc-467e-a16e-391bcefff510 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:58.752491 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cda40d-29fc-467e-a16e-391bcefff510 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:58.752931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cda40d-29fc-467e-a16e-391bcefff510 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:58.764361 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cda40d-29fc-467e-a16e-391bcefff510 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:58.769388 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-c0cda40d-29fc-467e-a16e-391bcefff510 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:19:58.769388 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:58.769388 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:58.769388 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:58.769388 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:58.769388 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:58.769388 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:58.769388 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:58.769388 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:58.769388 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:58.769388 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:58.769388 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:58.769388 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:58.769388 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:58.769388 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:58.769388 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:58.770308 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:58.770308 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:58.770308 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:58.770308 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:58.770308 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:58.770308 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:58.770308 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:58.770308 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:58.770308 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:58.770308 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:58.770308 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:58.770308 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:58.770308 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:58.770308 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:58.770308 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:58.770308 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:58.771163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:58.771163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:58.771163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:58.771163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:58.771163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:58.771163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:58.771163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:58.771163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:58.771163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:58.771163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:58.771163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:19:58.771163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:58.773325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 300/612] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 10:19:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:58.779401 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110567b1-5b90-4705-bcd9-a5feaa10f81d None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:58.797799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cda40d-29fc-467e-a16e-391bcefff510 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:58.891787 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:58.915101 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:58.917011 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110567b1-5b90-4705-bcd9-a5feaa10f81d None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:58.918854 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cda40d-29fc-467e-a16e-391bcefff510 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:58.963487 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67929a03-8e55-4a90-8f74-97b8c6a977fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:58.964136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67929a03-8e55-4a90-8f74-97b8c6a977fe None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:58.964415 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67929a03-8e55-4a90-8f74-97b8c6a977fe None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:58.964687 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67929a03-8e55-4a90-8f74-97b8c6a977fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:58.968660 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ced5ce-a6ce-404c-84fe-86a374514c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:58.969279 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ced5ce-a6ce-404c-84fe-86a374514c6a None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:58.969558 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ced5ce-a6ce-404c-84fe-86a374514c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:58.969833 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ced5ce-a6ce-404c-84fe-86a374514c6a None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:58.970530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ced5ce-a6ce-404c-84fe-86a374514c6a None admin] RBAC: Adding query filter params (name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:58.970834 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ced5ce-a6ce-404c-84fe-86a374514c6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:58.977101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ced5ce-a6ce-404c-84fe-86a374514c6a None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:58.988028 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67929a03-8e55-4a90-8f74-97b8c6a977fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:19:58.993071 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-67929a03-8e55-4a90-8f74-97b8c6a977fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:19:58.993071 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:58.993071 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:58.993071 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:58.993071 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:58.993071 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:58.993071 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:58.993071 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:58.993071 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:58.993071 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:58.993071 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:58.993071 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:58.993071 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:19:58.993071 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:19:58.993071 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:19:58.993071 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:19:58.993999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:19:58.993999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:19:58.993999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:19:58.993999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:19:58.993999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:58.993999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:58.993999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:58.993999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:58.993999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:58.993999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:58.993999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:58.993999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:58.993999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:58.993999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:58.993999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:58.993999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:58.994884 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:58.994884 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:58.994884 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:58.994884 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:58.994884 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:19:58.994884 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:19:58.994884 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:19:58.994884 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:19:58.994884 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:19:58.994884 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:19:58.994884 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:19:58.994884 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:59.000848 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 301/613] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 10:19:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:19:59.003091 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 313/614] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 10:19:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:59.004140 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67929a03-8e55-4a90-8f74-97b8c6a977fe None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:59.009130 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ced5ce-a6ce-404c-84fe-86a374514c6a None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:59.158640 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ced5ce-a6ce-404c-84fe-86a374514c6a None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:59.206916 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797a942-04c2-4d68-a6b7-650409aa6711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:59.207576 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797a942-04c2-4d68-a6b7-650409aa6711 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:59.207857 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797a942-04c2-4d68-a6b7-650409aa6711 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:59.208134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797a942-04c2-4d68-a6b7-650409aa6711 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:59.211923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67929a03-8e55-4a90-8f74-97b8c6a977fe None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:59.229374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c797a942-04c2-4d68-a6b7-650409aa6711 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:59.231070 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c797a942-04c2-4d68-a6b7-650409aa6711 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:59.237856 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:59.257116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-c797a942-04c2-4d68-a6b7-650409aa6711 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:59.257116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:59.257116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:59.257116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:59.257116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:59.257116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:59.257116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:59.257116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:59.257116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:59.257116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:59.257116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:59.257116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:59.257116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:19:59.257116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:19:59.257116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:19:59.257116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:19:59.258141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:59.258141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:59.258141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:59.258141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:59.258141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:59.258141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:59.258141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:59.258141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:59.258141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:59.258141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:59.258141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:59.258141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:59.258141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:59.258141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:59.258141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:59.258141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:59.258141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:19:59.259028 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:19:59.259028 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:59.259028 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:59.261190 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 314/615] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:19:59.268590 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797a942-04c2-4d68-a6b7-650409aa6711 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:59.299269 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87cf255-5c16-4502-b6ac-362b6aca795c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:59.299887 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cf255-5c16-4502-b6ac-362b6aca795c None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:59.300167 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cf255-5c16-4502-b6ac-362b6aca795c None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:59.300442 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cf255-5c16-4502-b6ac-362b6aca795c None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:59.301125 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87cf255-5c16-4502-b6ac-362b6aca795c None admin] RBAC: Adding query filter params (name=service) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:59.301413 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87cf255-5c16-4502-b6ac-362b6aca795c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:59.302807 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87cf255-5c16-4502-b6ac-362b6aca795c None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:59.324381 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 302/616] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 10:19:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:59.334067 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87cf255-5c16-4502-b6ac-362b6aca795c None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:59.380452 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797a942-04c2-4d68-a6b7-650409aa6711 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:59.400959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:59.421001 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e55af9-8925-45c8-b19b-2344a7dbae44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:59.422202 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e55af9-8925-45c8-b19b-2344a7dbae44 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:59.422707 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e55af9-8925-45c8-b19b-2344a7dbae44 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:59.422983 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e55af9-8925-45c8-b19b-2344a7dbae44 None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:59.423636 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e55af9-8925-45c8-b19b-2344a7dbae44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:59.423941 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e55af9-8925-45c8-b19b-2344a7dbae44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:59.428231 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e55af9-8925-45c8-b19b-2344a7dbae44 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:59.453690 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 315/617] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:19:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:19:59.464802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e55af9-8925-45c8-b19b-2344a7dbae44 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:59.566438 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87cf255-5c16-4502-b6ac-362b6aca795c None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:59.576718 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:59.611403 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb38d36-cc9a-4f00-9374-fcb6f827f6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:59.612034 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb38d36-cc9a-4f00-9374-fcb6f827f6d4 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:59.612307 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb38d36-cc9a-4f00-9374-fcb6f827f6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:59.612859 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb38d36-cc9a-4f00-9374-fcb6f827f6d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:59.625479 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb38d36-cc9a-4f00-9374-fcb6f827f6d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:59.626995 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb38d36-cc9a-4f00-9374-fcb6f827f6d4 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:59.643570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-acb38d36-cc9a-4f00-9374-fcb6f827f6d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:59.643570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:59.643570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:59.643570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:59.643570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:59.643570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:59.643570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:59.643570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:59.643570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:59.643570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:59.643570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:59.643570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:59.643570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:19:59.643570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:19:59.643570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:19:59.643570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:19:59.644567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:59.644567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:59.644567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:59.644567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:59.644567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:59.644567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:59.644567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:59.644567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:59.644567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:59.644567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:59.644567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:59.644567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:59.644567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:59.644567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:59.644567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:59.644567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:59.644567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:19:59.645443 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:19:59.645443 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:19:59.645443 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:19:59.645443 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 303/618] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:19:59.651641 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e55af9-8925-45c8-b19b-2344a7dbae44 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:59.666674 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb38d36-cc9a-4f00-9374-fcb6f827f6d4 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:59.697783 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2075096-edff-47ed-9ec9-048ad843f76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:59.698525 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2075096-edff-47ed-9ec9-048ad843f76e None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:59.698810 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2075096-edff-47ed-9ec9-048ad843f76e None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:59.699085 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2075096-edff-47ed-9ec9-048ad843f76e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:59.715134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2075096-edff-47ed-9ec9-048ad843f76e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:59.717403 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2075096-edff-47ed-9ec9-048ad843f76e None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:59.735378 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-f2075096-edff-47ed-9ec9-048ad843f76e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 26 10:19:59.735378 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:19:59.735378 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:19:59.735378 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:19:59.735378 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:19:59.735378 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:19:59.735378 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:19:59.735378 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:19:59.735378 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:19:59.735378 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:19:59.735378 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:19:59.735378 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:19:59.735378 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:19:59.735378 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:19:59.735378 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:19:59.735378 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:19:59.737432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:19:59.737432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:19:59.737432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:19:59.737432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:19:59.737432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:19:59.737432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:19:59.737432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:19:59.737432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:19:59.737432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:19:59.737432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:19:59.737432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:19:59.737432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:19:59.737432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:19:59.737432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:19:59.737432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:19:59.737432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:19:59.737432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:19:59.737432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:19:59.739799 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:19:59.739799 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:19:59.739799 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:19:59.739799 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:19:59.739799 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:19:59.739799 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:19:59.739799 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:19:59.739799 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:19:59.739799 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 26 10:19:59.739799 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:19:59.746426 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 316/619] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:19:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:19:59.750591 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2075096-edff-47ed-9ec9-048ad843f76e None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:59.819792 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:59.877647 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2075096-edff-47ed-9ec9-048ad843f76e None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:59.884039 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb38d36-cc9a-4f00-9374-fcb6f827f6d4 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:19:59.910416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:19:59.915237 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba340a63-e0e2-48bb-8f83-74e18e52565e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:59.915853 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba340a63-e0e2-48bb-8f83-74e18e52565e None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:59.916132 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba340a63-e0e2-48bb-8f83-74e18e52565e None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:59.916408 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba340a63-e0e2-48bb-8f83-74e18e52565e None admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:59.917145 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba340a63-e0e2-48bb-8f83-74e18e52565e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:59.917457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba340a63-e0e2-48bb-8f83-74e18e52565e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:59.921335 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba340a63-e0e2-48bb-8f83-74e18e52565e None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:59.956626 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 317/620] 10.222.0.49 () {56 vars in 1135 bytes} [Mon Sep 26 10:19:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:19:59.960498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8097b8-7148-428b-8ff4-8b06545acf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:19:59.961124 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8097b8-7148-428b-8ff4-8b06545acf7f None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:19:59.962329 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba340a63-e0e2-48bb-8f83-74e18e52565e None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:19:59.963259 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8097b8-7148-428b-8ff4-8b06545acf7f None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:19:59.963542 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8097b8-7148-428b-8ff4-8b06545acf7f None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:19:59.964194 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8097b8-7148-428b-8ff4-8b06545acf7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:19:59.964499 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8097b8-7148-428b-8ff4-8b06545acf7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:19:59.965874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8097b8-7148-428b-8ff4-8b06545acf7f None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:19:59.993751 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 304/621] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:19:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:00.011914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8097b8-7148-428b-8ff4-8b06545acf7f None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:00.082045 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba340a63-e0e2-48bb-8f83-74e18e52565e None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:00.115959 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b375c-dc66-40ae-b183-ef866182174c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:00.116593 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b375c-dc66-40ae-b183-ef866182174c None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:00.116871 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b375c-dc66-40ae-b183-ef866182174c None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:00.117143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b375c-dc66-40ae-b183-ef866182174c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:00.129092 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b375c-dc66-40ae-b183-ef866182174c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:00.130580 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b375c-dc66-40ae-b183-ef866182174c None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:00.146367 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-a95b375c-dc66-40ae-b183-ef866182174c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:00.146367 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:00.146367 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:00.146367 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:00.146367 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:00.146367 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:00.146367 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:00.146367 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:00.146367 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:00.146367 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:00.146367 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:00.146367 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:00.146367 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:00.146367 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:00.146367 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:00.146367 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:00.147362 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:00.147362 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:00.147362 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:00.147362 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:00.147362 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:00.147362 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:00.147362 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:00.147362 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:00.147362 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:00.147362 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:00.147362 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:00.147362 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:00.147362 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:00.147362 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:00.147362 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:00.147362 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:00.147362 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:00.148250 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:00.148250 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:00.148250 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:00.148250 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 318/622] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:19:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:00.155606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b375c-dc66-40ae-b183-ef866182174c None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:00.193568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:00.202231 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8097b8-7148-428b-8ff4-8b06545acf7f None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:00.265221 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ce28f-5263-4a3b-af6c-a3dd5418284a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:00.267442 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ce28f-5263-4a3b-af6c-a3dd5418284a None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:00.267728 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ce28f-5263-4a3b-af6c-a3dd5418284a None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:00.268005 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ce28f-5263-4a3b-af6c-a3dd5418284a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:00.282466 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:00.299028 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ce28f-5263-4a3b-af6c-a3dd5418284a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:00.300492 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ce28f-5263-4a3b-af6c-a3dd5418284a None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:00.303582 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b375c-dc66-40ae-b183-ef866182174c None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:00.322231 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-606ce28f-5263-4a3b-af6c-a3dd5418284a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 10:20:00.322231 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:00.322231 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:00.322231 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:00.322231 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:00.322231 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:00.322231 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:00.322231 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:00.322231 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:00.322231 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:00.322231 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:00.322231 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:00.322231 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:20:00.322231 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:20:00.322231 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:20:00.322231 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:20:00.323212 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:00.323212 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:00.323212 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:20:00.323212 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:00.323212 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:20:00.323212 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:00.323212 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:00.323212 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:00.323212 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:00.323212 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:00.323212 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:00.323212 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:00.323212 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:00.323212 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:00.323212 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:00.323212 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:00.323212 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:00.323212 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:00.324150 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:00.324150 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:00.324150 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:20:00.324150 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:20:00.324150 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:20:00.324150 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:20:00.324150 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:20:00.324150 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:20:00.324150 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 10:20:00.324150 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:00.324150 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 305/623] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Sep 26 10:20:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:20:00.343024 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ce28f-5263-4a3b-af6c-a3dd5418284a None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:00.345263 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bbfa9d-7a44-43dd-9778-781da03a75cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:00.345987 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bbfa9d-7a44-43dd-9778-781da03a75cc None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:00.346298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bbfa9d-7a44-43dd-9778-781da03a75cc None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:00.346603 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bbfa9d-7a44-43dd-9778-781da03a75cc None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:00.347328 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bbfa9d-7a44-43dd-9778-781da03a75cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:00.347635 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bbfa9d-7a44-43dd-9778-781da03a75cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:00.348997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bbfa9d-7a44-43dd-9778-781da03a75cc None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:00.362683 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 319/624] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:00.368259 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bbfa9d-7a44-43dd-9778-781da03a75cc None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:00.484681 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bbfa9d-7a44-43dd-9778-781da03a75cc None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:00.487419 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:00.524040 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a639a5fc-a78f-45dd-860c-f3ff8b320977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:00.524766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639a5fc-a78f-45dd-860c-f3ff8b320977 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:00.525048 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639a5fc-a78f-45dd-860c-f3ff8b320977 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:00.525722 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639a5fc-a78f-45dd-860c-f3ff8b320977 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:00.533631 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a639a5fc-a78f-45dd-860c-f3ff8b320977 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:00.535272 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a639a5fc-a78f-45dd-860c-f3ff8b320977 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:00.545204 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ce28f-5263-4a3b-af6c-a3dd5418284a None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:00.549016 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-a639a5fc-a78f-45dd-860c-f3ff8b320977 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:00.549016 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:00.549016 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:00.549016 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:00.549016 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:00.549016 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:00.549016 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:00.549016 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:00.549016 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:00.549016 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:00.549016 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:00.549016 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:00.549016 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:20:00.549016 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:20:00.549016 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:20:00.551004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:20:00.551004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:00.551004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:00.551004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:00.551004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:00.551004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:00.551004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:00.551004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:00.551004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:00.551004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:00.551004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:00.551004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:00.551004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:00.551004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:00.551004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:00.551004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:00.551004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:00.553067 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:20:00.553067 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:20:00.553067 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:20:00.553067 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:20:00.553067 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:20:00.553067 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:20:00.553067 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:00.553067 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:00.553067 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 320/625] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:20:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:00.553067 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7914158-6b19-4861-833f-7497b6c37132 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:00.553067 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7914158-6b19-4861-833f-7497b6c37132 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:00.554228 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7914158-6b19-4861-833f-7497b6c37132 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:00.554228 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 321/626] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:00.559704 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e0213-c2bd-4a74-8d76-028594e26a8e None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:00.560712 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e0213-c2bd-4a74-8d76-028594e26a8e None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:00.561085 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e0213-c2bd-4a74-8d76-028594e26a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:00.583630 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-ca2e0213-c2bd-4a74-8d76-028594e26a8e None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:00.585082 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-ca2e0213-c2bd-4a74-8d76-028594e26a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:00.586031 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a1339-f9c6-4745-a8c3-bd7f24f56962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:00.586682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a1339-f9c6-4745-a8c3-bd7f24f56962 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:00.587010 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a1339-f9c6-4745-a8c3-bd7f24f56962 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:00.587287 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a1339-f9c6-4745-a8c3-bd7f24f56962 None admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:00.588122 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a1339-f9c6-4745-a8c3-bd7f24f56962 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:00.588427 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a1339-f9c6-4745-a8c3-bd7f24f56962 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:00.589970 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a1339-f9c6-4745-a8c3-bd7f24f56962 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:00.616253 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 306/627] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Sep 26 10:20:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:00.617182 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a1339-f9c6-4745-a8c3-bd7f24f56962 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:00.666241 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 322/628] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:20:00.667360 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e0213-c2bd-4a74-8d76-028594e26a8e None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:00.684025 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:00.754444 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:00.781804 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e0213-c2bd-4a74-8d76-028594e26a8e None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:00.790944 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a1339-f9c6-4745-a8c3-bd7f24f56962 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:00.831845 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faee974-442b-4195-b8b1-1ff49fa758d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:00.832859 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faee974-442b-4195-b8b1-1ff49fa758d2 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:00.833142 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faee974-442b-4195-b8b1-1ff49fa758d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:00.833422 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faee974-442b-4195-b8b1-1ff49fa758d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:00.868003 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8496b19f-e583-4b2a-a58f-c184fefd119d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:00.868635 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496b19f-e583-4b2a-a58f-c184fefd119d None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:00.868914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496b19f-e583-4b2a-a58f-c184fefd119d None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:00.869184 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496b19f-e583-4b2a-a58f-c184fefd119d None admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:00.869892 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8496b19f-e583-4b2a-a58f-c184fefd119d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:00.870221 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8496b19f-e583-4b2a-a58f-c184fefd119d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:00.876973 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8496b19f-e583-4b2a-a58f-c184fefd119d None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:00.897738 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 307/629] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:20:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:00.905750 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8496b19f-e583-4b2a-a58f-c184fefd119d None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:01.002596 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faee974-442b-4195-b8b1-1ff49fa758d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:01.004043 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faee974-442b-4195-b8b1-1ff49fa758d2 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:01.036344 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8496b19f-e583-4b2a-a58f-c184fefd119d None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:01.049834 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:01.072801 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebe989-5ebf-4a3e-8992-e45f4966f8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:01.073456 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebe989-5ebf-4a3e-8992-e45f4966f8ab None admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:01.073733 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebe989-5ebf-4a3e-8992-e45f4966f8ab None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:01.074004 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebe989-5ebf-4a3e-8992-e45f4966f8ab None admin] PATH_INFO: `/v3/projects/cff3a6011f4c4893bddffdad3d69f3df/users/5e6be34fc93a4551a83e5b8380188a6b/roles/d36e58344b604bcda1cee49065e8497e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:01.089789 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:01.090666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebe989-5ebf-4a3e-8992-e45f4966f8ab None admin] RBAC: Authorizing `identity:create_grant(project_id=cff3a6011f4c4893bddffdad3d69f3df, user_id=5e6be34fc93a4551a83e5b8380188a6b, role_id=d36e58344b604bcda1cee49065e8497e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:01.092690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebe989-5ebf-4a3e-8992-e45f4966f8ab None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:01.108835 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 308/630] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 10:20:00 2022] PUT /identity/v3/projects/cff3a6011f4c4893bddffdad3d69f3df/users/5e6be34fc93a4551a83e5b8380188a6b/roles/d36e58344b604bcda1cee49065e8497e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:20:01.110244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa3d86-1209-4782-b083-9b2a2bb3297e None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:01.110531 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa3d86-1209-4782-b083-9b2a2bb3297e None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:01.110813 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa3d86-1209-4782-b083-9b2a2bb3297e None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:01.112393 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 309/631] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:01.113070 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa3d86-1209-4782-b083-9b2a2bb3297e None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:01.128497 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 323/632] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:20:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 10:20:01.129810 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d6dcf-4016-41bd-90fc-d8bd2c72e17b None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:01.130145 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d6dcf-4016-41bd-90fc-d8bd2c72e17b None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:01.130419 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d6dcf-4016-41bd-90fc-d8bd2c72e17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:01.151379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-276d6dcf-4016-41bd-90fc-d8bd2c72e17b None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:01.152597 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-276d6dcf-4016-41bd-90fc-d8bd2c72e17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:01.258128 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 324/633] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:20:01.258128 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d6dcf-4016-41bd-90fc-d8bd2c72e17b None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:01.269013 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa3d86-1209-4782-b083-9b2a2bb3297e None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:01.319853 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca67c3-84fc-424b-9136-51c3f3268af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:01.320821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca67c3-84fc-424b-9136-51c3f3268af8 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:01.321051 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca67c3-84fc-424b-9136-51c3f3268af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:01.321456 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca67c3-84fc-424b-9136-51c3f3268af8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:01.335963 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca67c3-84fc-424b-9136-51c3f3268af8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:20:01.342672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-46ca67c3-84fc-424b-9136-51c3f3268af8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:20:01.342672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:01.342672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:01.342672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:01.342672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:01.342672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:01.342672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:01.342672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:01.342672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:01.342672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:01.342672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:01.342672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:01.342672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:20:01.342672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:20:01.342672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:20:01.342672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:20:01.343618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:20:01.343618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:20:01.343618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:20:01.343618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:20:01.343618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:01.343618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:01.343618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:01.343618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:01.343618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:01.343618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:01.343618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:01.343618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:01.343618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:01.343618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:01.343618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:01.343618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:01.344470 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:01.344470 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:01.344470 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:01.344470 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:01.344470 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:20:01.344470 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:20:01.344470 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:20:01.344470 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:20:01.344470 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:20:01.344470 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:20:01.344470 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:20:01.344470 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:01.345397 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 310/634] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 10:20:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:01.346213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a9636-0f6a-4a68-8f29-ed3b14ab7160 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:01.346742 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a9636-0f6a-4a68-8f29-ed3b14ab7160 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:01.346867 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a9636-0f6a-4a68-8f29-ed3b14ab7160 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:01.348864 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 311/635] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:01.359295 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2dd194-ce75-488b-a845-833c05afc930 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:01.359567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2dd194-ce75-488b-a845-833c05afc930 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:01.359838 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2dd194-ce75-488b-a845-833c05afc930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:01.371134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d6dcf-4016-41bd-90fc-d8bd2c72e17b None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:01.378813 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-0e2dd194-ce75-488b-a845-833c05afc930 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:01.380170 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:01.381010 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-0e2dd194-ce75-488b-a845-833c05afc930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:01.405580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:01.411294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a8f659-1e00-40e1-9a73-039764cadf96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:01.412233 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8f659-1e00-40e1-9a73-039764cadf96 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:01.412713 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8f659-1e00-40e1-9a73-039764cadf96 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:01.413187 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8f659-1e00-40e1-9a73-039764cadf96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:01.425726 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a8f659-1e00-40e1-9a73-039764cadf96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:01.427467 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a8f659-1e00-40e1-9a73-039764cadf96 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:01.442800 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-16a8f659-1e00-40e1-9a73-039764cadf96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:01.442800 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:01.442800 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:01.442800 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:01.442800 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:01.442800 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:01.442800 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:01.442800 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:01.442800 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:01.442800 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:01.442800 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:01.442800 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:01.442800 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:01.442800 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:01.442800 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:01.442800 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:01.443807 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:01.443807 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:01.443807 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:01.443807 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:01.443807 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:01.443807 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:01.443807 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:01.443807 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:01.443807 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:01.443807 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:01.443807 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:01.443807 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:01.443807 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:01.443807 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:01.443807 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:01.443807 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:01.443807 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:01.444719 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:01.444719 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:01.444719 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:01.446587 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 325/636] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:01.447528 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a8f659-1e00-40e1-9a73-039764cadf96 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:01.471707 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 312/637] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:20:01.472860 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2dd194-ce75-488b-a845-833c05afc930 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:01.599698 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a8f659-1e00-40e1-9a73-039764cadf96 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:01.612814 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2dd194-ce75-488b-a845-833c05afc930 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:01.638993 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6228c3f9-5a57-4de3-a618-d38f7e77e1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:01.639703 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228c3f9-5a57-4de3-a618-d38f7e77e1e1 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:01.640049 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228c3f9-5a57-4de3-a618-d38f7e77e1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:01.640392 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228c3f9-5a57-4de3-a618-d38f7e77e1e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:01.641119 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6228c3f9-5a57-4de3-a618-d38f7e77e1e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:01.641567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6228c3f9-5a57-4de3-a618-d38f7e77e1e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:01.643317 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6228c3f9-5a57-4de3-a618-d38f7e77e1e1 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:01.655887 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c565c3-19ef-4984-a515-25d61ed11e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:01.656492 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c565c3-19ef-4984-a515-25d61ed11e4e None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:01.656767 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c565c3-19ef-4984-a515-25d61ed11e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:01.657037 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c565c3-19ef-4984-a515-25d61ed11e4e None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:01.657679 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c565c3-19ef-4984-a515-25d61ed11e4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:01.657979 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c565c3-19ef-4984-a515-25d61ed11e4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:01.658218 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 326/638] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 10:20:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:01.659146 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6228c3f9-5a57-4de3-a618-d38f7e77e1e1 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:01.659369 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c565c3-19ef-4984-a515-25d61ed11e4e None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:01.672748 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 313/639] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:01.673576 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c565c3-19ef-4984-a515-25d61ed11e4e None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:01.686198 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:01.698426 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:01.786587 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6228c3f9-5a57-4de3-a618-d38f7e77e1e1 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:01.805138 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c565c3-19ef-4984-a515-25d61ed11e4e None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:01.826699 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78960613-5513-47db-b70c-6cceb2699c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:01.827335 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78960613-5513-47db-b70c-6cceb2699c9b None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:01.827613 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78960613-5513-47db-b70c-6cceb2699c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:01.827885 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78960613-5513-47db-b70c-6cceb2699c9b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:01.839389 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78960613-5513-47db-b70c-6cceb2699c9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:20:01.843803 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c00bac-abf1-411e-9b32-f8c56904f373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:01.844435 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c00bac-abf1-411e-9b32-f8c56904f373 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:01.844711 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c00bac-abf1-411e-9b32-f8c56904f373 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:01.844952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-78960613-5513-47db-b70c-6cceb2699c9b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:20:01.844952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:01.844952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:01.844952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:01.844952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:01.844952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:01.844952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:01.844952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:01.844952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:01.844952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:01.844952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:01.844952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:01.844952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:20:01.844952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:20:01.844952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:20:01.844952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:20:01.846010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:20:01.846010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:20:01.846010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:20:01.846010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:20:01.846010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:01.846010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:01.846010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:01.846010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:01.846010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:01.846010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:01.846010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:01.846010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:01.846010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:01.846010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:01.846010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:01.846010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:01.846931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:01.846931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:01.846931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:01.846931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:01.846931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:20:01.846931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:20:01.846931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:20:01.846931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:20:01.846931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:20:01.846931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:20:01.846931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:20:01.846931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:01.847606 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c00bac-abf1-411e-9b32-f8c56904f373 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:01.847683 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 327/640] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 10:20:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:01.847683 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78960613-5513-47db-b70c-6cceb2699c9b None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:01.860600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c00bac-abf1-411e-9b32-f8c56904f373 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:01.862009 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c00bac-abf1-411e-9b32-f8c56904f373 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:01.877366 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-e1c00bac-abf1-411e-9b32-f8c56904f373 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:01.877366 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:01.877366 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:01.877366 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:01.877366 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:01.877366 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:01.877366 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:01.877366 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:01.877366 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:01.877366 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:01.877366 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:01.877366 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:01.877366 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:01.877366 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:01.877366 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:01.877366 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:01.878557 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:01.878557 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:01.878557 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:01.878557 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:01.878557 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:01.878557 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:01.878557 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:01.878557 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:01.878557 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:01.878557 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:01.878557 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:01.878557 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:01.878557 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:01.878557 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:01.878557 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:01.878557 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:01.878557 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:01.879598 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:01.879598 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:01.879598 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:01.879598 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 314/641] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:01.883809 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c00bac-abf1-411e-9b32-f8c56904f373 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:01.943091 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:01.948682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:01.965328 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78960613-5513-47db-b70c-6cceb2699c9b None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:01.988943 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c00bac-abf1-411e-9b32-f8c56904f373 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:02.002337 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b415969-0532-484e-938c-14add16688a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:02.003252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b415969-0532-484e-938c-14add16688a6 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:02.003252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b415969-0532-484e-938c-14add16688a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:02.003556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b415969-0532-484e-938c-14add16688a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:02.010673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b415969-0532-484e-938c-14add16688a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:02.012149 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b415969-0532-484e-938c-14add16688a6 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:02.023119 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc2e99e-9265-401c-a216-730ce7975d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:02.023741 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2e99e-9265-401c-a216-730ce7975d77 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:02.024016 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2e99e-9265-401c-a216-730ce7975d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:02.024283 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2e99e-9265-401c-a216-730ce7975d77 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:02.024450 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-2b415969-0532-484e-938c-14add16688a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:02.024450 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:02.024450 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:02.024450 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:02.024450 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:02.024450 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:02.024450 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:02.024450 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:02.024450 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:02.024450 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:02.024450 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:02.024450 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:02.024450 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:20:02.024450 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:20:02.024450 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:20:02.025373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:20:02.025373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:02.025373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:02.025373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:02.025373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:02.025373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:02.025373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:02.025373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:02.025373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:02.025373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:02.025373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:02.025373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:02.025373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:02.025373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:02.025373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:02.025373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:02.025373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:02.026773 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:20:02.026773 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:20:02.026773 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:20:02.026773 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:20:02.026773 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:20:02.026773 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:20:02.026773 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:02.026773 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:02.026773 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 328/642] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:20:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:02.026773 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b415969-0532-484e-938c-14add16688a6 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:02.040588 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc2e99e-9265-401c-a216-730ce7975d77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:20:02.045940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-4dc2e99e-9265-401c-a216-730ce7975d77 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:20:02.045940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:02.045940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:02.045940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:02.045940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:02.045940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:02.045940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:02.045940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:02.045940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:02.045940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:02.045940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:02.045940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:02.045940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:20:02.045940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:20:02.045940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:20:02.045940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:20:02.046952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:20:02.046952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:20:02.046952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:20:02.046952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:20:02.046952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:02.046952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:02.046952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:02.046952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:02.046952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:02.046952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:02.046952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:02.046952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:02.046952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:02.046952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:02.046952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:02.046952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:02.047857 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:02.047857 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:02.047857 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:02.047857 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:02.047857 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:20:02.047857 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:20:02.047857 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:20:02.047857 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:20:02.047857 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:20:02.047857 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:20:02.047857 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:20:02.047857 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:02.047857 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 315/643] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 10:20:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:02.049486 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc2e99e-9265-401c-a216-730ce7975d77 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:02.142363 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b415969-0532-484e-938c-14add16688a6 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:02.157911 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc2e99e-9265-401c-a216-730ce7975d77 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:02.178289 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2bd174-2d16-4846-b3d2-49f8cae7fb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:02.178946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2bd174-2d16-4846-b3d2-49f8cae7fb5c None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:02.179488 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2bd174-2d16-4846-b3d2-49f8cae7fb5c None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:02.179763 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2bd174-2d16-4846-b3d2-49f8cae7fb5c None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:02.180527 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2bd174-2d16-4846-b3d2-49f8cae7fb5c None admin] RBAC: Adding query filter params (name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:02.180828 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2bd174-2d16-4846-b3d2-49f8cae7fb5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:02.181241 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:02.183314 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2bd174-2d16-4846-b3d2-49f8cae7fb5c None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:02.192966 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349e600-b44d-4e59-a576-3399788331e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:02.197988 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 329/644] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 10:20:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:02.198171 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349e600-b44d-4e59-a576-3399788331e4 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:02.198171 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349e600-b44d-4e59-a576-3399788331e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:02.198171 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349e600-b44d-4e59-a576-3399788331e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:02.198430 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2bd174-2d16-4846-b3d2-49f8cae7fb5c None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:02.198514 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349e600-b44d-4e59-a576-3399788331e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:02.198514 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349e600-b44d-4e59-a576-3399788331e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:02.198514 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349e600-b44d-4e59-a576-3399788331e4 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:02.201213 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:02.210467 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 316/645] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:02.211123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349e600-b44d-4e59-a576-3399788331e4 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:02.318916 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2bd174-2d16-4846-b3d2-49f8cae7fb5c None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:02.328948 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349e600-b44d-4e59-a576-3399788331e4 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:02.356167 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8970c8-0a53-46e2-b904-66ba33973eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:02.357271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8970c8-0a53-46e2-b904-66ba33973eaa None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:02.357719 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8970c8-0a53-46e2-b904-66ba33973eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:02.357997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8970c8-0a53-46e2-b904-66ba33973eaa None admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:02.359064 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8970c8-0a53-46e2-b904-66ba33973eaa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:02.359377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8970c8-0a53-46e2-b904-66ba33973eaa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:02.360862 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8970c8-0a53-46e2-b904-66ba33973eaa None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:02.365949 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caaa157-3ceb-4987-86bb-3dbe54a365bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:02.366575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caaa157-3ceb-4987-86bb-3dbe54a365bc None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:02.366853 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caaa157-3ceb-4987-86bb-3dbe54a365bc None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:02.367120 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caaa157-3ceb-4987-86bb-3dbe54a365bc None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:02.368349 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caaa157-3ceb-4987-86bb-3dbe54a365bc None admin] RBAC: Adding query filter params (name=service) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:02.368652 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caaa157-3ceb-4987-86bb-3dbe54a365bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:02.370002 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caaa157-3ceb-4987-86bb-3dbe54a365bc None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:02.378159 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 330/646] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:20:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:02.378159 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8970c8-0a53-46e2-b904-66ba33973eaa None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:02.381929 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 317/647] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 10:20:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:02.383021 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caaa157-3ceb-4987-86bb-3dbe54a365bc None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:02.437333 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:02.452317 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:02.492365 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8970c8-0a53-46e2-b904-66ba33973eaa None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:02.497769 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caaa157-3ceb-4987-86bb-3dbe54a365bc None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:02.526683 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20cff73-ad94-4f33-9efc-8b34b1d1419f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:02.527301 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20cff73-ad94-4f33-9efc-8b34b1d1419f None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:02.527575 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20cff73-ad94-4f33-9efc-8b34b1d1419f None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:02.527841 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20cff73-ad94-4f33-9efc-8b34b1d1419f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:02.533843 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789d1f2-9b08-4bc1-ab48-4b6cbb748acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:02.534729 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789d1f2-9b08-4bc1-ab48-4b6cbb748acf None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:02.535175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789d1f2-9b08-4bc1-ab48-4b6cbb748acf None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:02.535740 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789d1f2-9b08-4bc1-ab48-4b6cbb748acf None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:02.537575 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20cff73-ad94-4f33-9efc-8b34b1d1419f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:02.539073 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20cff73-ad94-4f33-9efc-8b34b1d1419f None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:02.545017 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1789d1f2-9b08-4bc1-ab48-4b6cbb748acf None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:02.546613 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1789d1f2-9b08-4bc1-ab48-4b6cbb748acf None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:02.554225 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-c20cff73-ad94-4f33-9efc-8b34b1d1419f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:02.554225 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:02.554225 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:02.554225 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:02.554225 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:02.554225 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:02.554225 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:02.554225 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:02.554225 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:02.554225 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:02.554225 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:02.554225 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:02.554225 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:02.554225 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:02.554225 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:02.554225 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:02.555214 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:02.555214 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:02.555214 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:02.555214 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:02.555214 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:02.555214 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:02.555214 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:02.555214 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:02.555214 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:02.555214 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:02.555214 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:02.555214 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:02.555214 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:02.555214 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:02.555214 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:02.555214 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:02.555214 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:02.556114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:02.556114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:02.556114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:02.557945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-1789d1f2-9b08-4bc1-ab48-4b6cbb748acf None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 26 10:20:02.557945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:02.557945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:02.557945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:02.557945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:02.557945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:02.557945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:02.557945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:02.557945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:02.557945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:02.557945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:02.557945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:02.557945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:20:02.557945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:20:02.557945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:20:02.557945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:20:02.558872 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:02.558872 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:02.558872 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:20:02.558872 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:02.558872 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:20:02.558872 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:02.558872 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:02.558872 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:02.558872 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:02.558872 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:02.558872 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:02.558872 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:02.558872 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:02.558872 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:02.558872 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:02.558872 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:02.558872 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:02.558872 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:02.559896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:02.559896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:02.559896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:20:02.559896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:20:02.559896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:20:02.559896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:20:02.559896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:20:02.559896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:20:02.559896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 26 10:20:02.559896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:02.561577 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 331/648] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:02.561577 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20cff73-ad94-4f33-9efc-8b34b1d1419f None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:02.561994 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 318/649] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 10:20:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:02.562888 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789d1f2-9b08-4bc1-ab48-4b6cbb748acf None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:02.666661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789d1f2-9b08-4bc1-ab48-4b6cbb748acf None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:02.669801 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20cff73-ad94-4f33-9efc-8b34b1d1419f None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:02.674543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:02.685094 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:02.703820 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57d9a9-7af3-4020-a195-9c7346b1bad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:02.704390 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57d9a9-7af3-4020-a195-9c7346b1bad6 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:02.704677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57d9a9-7af3-4020-a195-9c7346b1bad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:02.704949 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57d9a9-7af3-4020-a195-9c7346b1bad6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:02.710173 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b901c-0b86-4043-96a9-abf850c8f118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:02.710776 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b901c-0b86-4043-96a9-abf850c8f118 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:02.711057 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b901c-0b86-4043-96a9-abf850c8f118 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:02.711331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b901c-0b86-4043-96a9-abf850c8f118 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:02.712049 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783b901c-0b86-4043-96a9-abf850c8f118 None admin] RBAC: Adding query filter params (role.id=d36e58344b604bcda1cee49065e8497e, scope.project.id=cff3a6011f4c4893bddffdad3d69f3df, user.id=cee41014d5d043888c6b399710bccab4) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:02.712357 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783b901c-0b86-4043-96a9-abf850c8f118 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:02.714232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783b901c-0b86-4043-96a9-abf850c8f118 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:02.718064 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57d9a9-7af3-4020-a195-9c7346b1bad6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:02.719653 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57d9a9-7af3-4020-a195-9c7346b1bad6 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:02.721171 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 332/650] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:20:02 2022] GET /identity/v3/role_assignments?user.id=cee41014d5d043888c6b399710bccab4&scope.project.id=cff3a6011f4c4893bddffdad3d69f3df&role.id=d36e58344b604bcda1cee49065e8497e => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:02.721968 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b901c-0b86-4043-96a9-abf850c8f118 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:02.746974 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-6a57d9a9-7af3-4020-a195-9c7346b1bad6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:02.746974 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:02.746974 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:02.746974 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:02.746974 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:02.746974 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:02.746974 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:02.746974 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:02.746974 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:02.746974 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:02.746974 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:02.746974 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:02.746974 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:02.746974 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:02.746974 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:02.746974 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:02.748744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:02.748744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:02.748744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:02.748744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:02.748744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:02.748744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:02.748744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:02.748744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:02.748744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:02.748744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:02.748744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:02.748744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:02.748744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:02.748744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:02.748744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:02.748744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:02.748744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:02.749815 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:02.749815 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:02.749815 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:02.751487 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 319/651] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:02.753291 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57d9a9-7af3-4020-a195-9c7346b1bad6 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:02.865858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b901c-0b86-4043-96a9-abf850c8f118 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:02.896683 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57d9a9-7af3-4020-a195-9c7346b1bad6 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:02.909259 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb670e1-e5a9-4db0-909e-4b08e2c777a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:02.910398 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb670e1-e5a9-4db0-909e-4b08e2c777a4 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:02.910819 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb670e1-e5a9-4db0-909e-4b08e2c777a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:02.911230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb670e1-e5a9-4db0-909e-4b08e2c777a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:02.912127 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb670e1-e5a9-4db0-909e-4b08e2c777a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:02.912593 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb670e1-e5a9-4db0-909e-4b08e2c777a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:02.914992 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb670e1-e5a9-4db0-909e-4b08e2c777a4 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:02.930702 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 333/652] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:02.931885 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb670e1-e5a9-4db0-909e-4b08e2c777a4 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:02.943838 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29bced1-deb0-44bc-bb29-24e8a23966db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:02.944454 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29bced1-deb0-44bc-bb29-24e8a23966db None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:02.944735 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29bced1-deb0-44bc-bb29-24e8a23966db None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:02.945009 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29bced1-deb0-44bc-bb29-24e8a23966db None admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:02.945794 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29bced1-deb0-44bc-bb29-24e8a23966db None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:02.946143 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29bced1-deb0-44bc-bb29-24e8a23966db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:02.947578 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29bced1-deb0-44bc-bb29-24e8a23966db None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:02.969875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 320/653] 10.222.0.49 () {56 vars in 1139 bytes} [Mon Sep 26 10:20:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:02.970781 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29bced1-deb0-44bc-bb29-24e8a23966db None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:02.989697 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:02.992137 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:03.073424 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb670e1-e5a9-4db0-909e-4b08e2c777a4 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:03.089477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29bced1-deb0-44bc-bb29-24e8a23966db None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:03.118982 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ebc589-98e9-4c09-90ed-b285cd3f1bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:03.119613 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ebc589-98e9-4c09-90ed-b285cd3f1bbe None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:03.119890 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ebc589-98e9-4c09-90ed-b285cd3f1bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:03.120162 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ebc589-98e9-4c09-90ed-b285cd3f1bbe None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:03.120802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ebc589-98e9-4c09-90ed-b285cd3f1bbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:03.121268 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ebc589-98e9-4c09-90ed-b285cd3f1bbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:03.123278 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ebc589-98e9-4c09-90ed-b285cd3f1bbe None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:03.129687 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb011f-2246-430c-894f-ff9a6288c8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:03.130368 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb011f-2246-430c-894f-ff9a6288c8c5 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:03.130655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb011f-2246-430c-894f-ff9a6288c8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:03.130932 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb011f-2246-430c-894f-ff9a6288c8c5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:03.136746 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 334/654] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:03.138400 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ebc589-98e9-4c09-90ed-b285cd3f1bbe None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:03.138886 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb011f-2246-430c-894f-ff9a6288c8c5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:03.140470 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb011f-2246-430c-894f-ff9a6288c8c5 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:03.152709 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-f4bb011f-2246-430c-894f-ff9a6288c8c5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 26 10:20:03.152709 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:03.152709 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:03.152709 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:03.152709 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:03.152709 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:03.152709 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:03.152709 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:03.152709 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:03.152709 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:03.152709 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:03.152709 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:03.152709 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:20:03.152709 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:20:03.152709 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:20:03.152709 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:20:03.153762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:03.153762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:03.153762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:20:03.153762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:03.153762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:20:03.153762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:03.153762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:03.153762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:03.153762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:03.153762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:03.153762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:03.153762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:03.153762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:03.153762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:03.153762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:03.153762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:03.153762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:03.153762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:03.154765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:03.154765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:03.154765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:20:03.154765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:20:03.154765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:20:03.154765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:20:03.154765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:20:03.154765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:20:03.154765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 26 10:20:03.154765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:03.158160 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 321/655] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:20:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:03.159036 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb011f-2246-430c-894f-ff9a6288c8c5 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:03.253243 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:03.258260 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:03.259024 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ebc589-98e9-4c09-90ed-b285cd3f1bbe None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:03.269060 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb011f-2246-430c-894f-ff9a6288c8c5 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:03.300279 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861df2a7-3f74-48b5-a4f8-f079313bd4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:03.301380 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861df2a7-3f74-48b5-a4f8-f079313bd4b4 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:03.301825 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861df2a7-3f74-48b5-a4f8-f079313bd4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:03.302351 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861df2a7-3f74-48b5-a4f8-f079313bd4b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:03.305503 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d54172c-67f8-4f64-b8cb-a3bbfdd8bb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:03.306428 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54172c-67f8-4f64-b8cb-a3bbfdd8bb3c None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:03.306934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54172c-67f8-4f64-b8cb-a3bbfdd8bb3c None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:03.307410 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54172c-67f8-4f64-b8cb-a3bbfdd8bb3c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:03.316515 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d54172c-67f8-4f64-b8cb-a3bbfdd8bb3c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:03.318316 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d54172c-67f8-4f64-b8cb-a3bbfdd8bb3c None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:03.330897 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-9d54172c-67f8-4f64-b8cb-a3bbfdd8bb3c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 10:20:03.330897 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:03.330897 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:03.330897 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:03.330897 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:03.330897 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:03.330897 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:03.330897 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:03.330897 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:03.330897 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:03.330897 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:03.330897 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:03.330897 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:20:03.330897 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:20:03.330897 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:20:03.330897 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:20:03.332575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:03.332575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:03.332575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:20:03.332575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:03.332575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:20:03.332575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:03.332575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:03.332575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:03.332575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:03.332575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:03.332575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:03.332575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:03.332575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:03.332575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:03.332575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:03.332575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:03.332575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:03.332575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:03.333537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:03.333537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:03.333537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:20:03.333537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:20:03.333537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:20:03.333537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:20:03.333537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:20:03.333537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:20:03.333537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 10:20:03.333537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:03.336131 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 322/656] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:20:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:03.337213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d54172c-67f8-4f64-b8cb-a3bbfdd8bb3c None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:03.378085 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861df2a7-3f74-48b5-a4f8-f079313bd4b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:03.379518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861df2a7-3f74-48b5-a4f8-f079313bd4b4 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:03.446009 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d54172c-67f8-4f64-b8cb-a3bbfdd8bb3c None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:03.482807 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c379f4-2bd4-40c8-bdad-83274dccf266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:03.483408 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c379f4-2bd4-40c8-bdad-83274dccf266 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:03.483683 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c379f4-2bd4-40c8-bdad-83274dccf266 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:03.483987 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c379f4-2bd4-40c8-bdad-83274dccf266 None admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:03.484665 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c379f4-2bd4-40c8-bdad-83274dccf266 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:03.484963 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c379f4-2bd4-40c8-bdad-83274dccf266 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:03.485662 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:03.486590 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c379f4-2bd4-40c8-bdad-83274dccf266 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:03.489703 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:03.496058 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 335/657] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:20:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 10:20:03.497162 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861df2a7-3f74-48b5-a4f8-f079313bd4b4 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:03.505717 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 323/658] 10.222.0.49 () {56 vars in 1137 bytes} [Mon Sep 26 10:20:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:03.507191 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dcee7c-e010-412f-9190-670a6d87dd11 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:03.507471 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dcee7c-e010-412f-9190-670a6d87dd11 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:03.507737 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dcee7c-e010-412f-9190-670a6d87dd11 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:03.513065 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 324/659] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:03.513065 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dcee7c-e010-412f-9190-670a6d87dd11 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:03.611846 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861df2a7-3f74-48b5-a4f8-f079313bd4b4 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:03.618188 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dcee7c-e010-412f-9190-670a6d87dd11 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:03.647896 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e316777-1cb5-4078-80e8-01d66aedd00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:03.648502 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e316777-1cb5-4078-80e8-01d66aedd00f None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:03.648780 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e316777-1cb5-4078-80e8-01d66aedd00f None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:03.649051 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e316777-1cb5-4078-80e8-01d66aedd00f None admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:03.649773 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e316777-1cb5-4078-80e8-01d66aedd00f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:03.650191 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e316777-1cb5-4078-80e8-01d66aedd00f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:03.651548 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e316777-1cb5-4078-80e8-01d66aedd00f None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:03.654732 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7853d9b-ab71-4b9d-ad17-f8028a239e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:03.655353 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7853d9b-ab71-4b9d-ad17-f8028a239e8c None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:03.655624 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7853d9b-ab71-4b9d-ad17-f8028a239e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:03.655889 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7853d9b-ab71-4b9d-ad17-f8028a239e8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:03.666050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7853d9b-ab71-4b9d-ad17-f8028a239e8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:03.667522 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7853d9b-ab71-4b9d-ad17-f8028a239e8c None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:03.676636 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 336/660] 10.222.0.49 () {56 vars in 1137 bytes} [Mon Sep 26 10:20:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:03.678345 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e316777-1cb5-4078-80e8-01d66aedd00f None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:03.684649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-a7853d9b-ab71-4b9d-ad17-f8028a239e8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:03.684649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:03.684649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:03.684649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:03.684649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:03.684649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:03.684649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:03.684649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:03.684649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:03.684649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:03.684649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:03.684649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:03.684649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:03.684649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:03.684649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:03.684649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:03.685618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:03.685618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:03.685618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:03.685618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:03.685618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:03.685618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:03.685618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:03.685618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:03.685618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:03.685618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:03.685618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:03.685618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:03.685618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:03.685618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:03.685618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:03.685618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:03.685618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:03.686543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:03.686543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:03.686543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:03.690075 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 325/661] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:03.691489 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac6ee5-ee4a-4b46-a554-d1ad3d9f513d None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:03.691768 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac6ee5-ee4a-4b46-a554-d1ad3d9f513d None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:03.692033 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac6ee5-ee4a-4b46-a554-d1ad3d9f513d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:03.709992 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-a4ac6ee5-ee4a-4b46-a554-d1ad3d9f513d None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:03.711238 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-a4ac6ee5-ee4a-4b46-a554-d1ad3d9f513d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:03.740549 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:03.755612 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:03.782066 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 326/662] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:20:03.782066 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac6ee5-ee4a-4b46-a554-d1ad3d9f513d None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:03.804850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e316777-1cb5-4078-80e8-01d66aedd00f None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:03.846512 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8671f-d8bc-47b6-865a-b5c8ce6a9bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:03.847360 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8671f-d8bc-47b6-865a-b5c8ce6a9bbc None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:03.847787 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8671f-d8bc-47b6-865a-b5c8ce6a9bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:03.848204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8671f-d8bc-47b6-865a-b5c8ce6a9bbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:03.918617 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac6ee5-ee4a-4b46-a554-d1ad3d9f513d None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:03.938467 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8671f-d8bc-47b6-865a-b5c8ce6a9bbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:03.940920 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8671f-d8bc-47b6-865a-b5c8ce6a9bbc None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:03.955880 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a607324-b74b-4be2-9f71-ed9a7af26a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:03.956589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a607324-b74b-4be2-9f71-ed9a7af26a13 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:03.956932 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a607324-b74b-4be2-9f71-ed9a7af26a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:03.957271 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a607324-b74b-4be2-9f71-ed9a7af26a13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:04.004270 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:04.020569 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:04.055476 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a607324-b74b-4be2-9f71-ed9a7af26a13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:04.057204 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a607324-b74b-4be2-9f71-ed9a7af26a13 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:04.058519 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 337/663] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:20:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 10:20:04.059535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8671f-d8bc-47b6-865a-b5c8ce6a9bbc None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:04.162931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 327/664] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:20:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 10:20:04.163875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a607324-b74b-4be2-9f71-ed9a7af26a13 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:04.169984 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8671f-d8bc-47b6-865a-b5c8ce6a9bbc None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:04.205541 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993954c-e8a3-4d53-b4b9-7fa7386fc3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:04.206433 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993954c-e8a3-4d53-b4b9-7fa7386fc3b1 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:04.206708 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993954c-e8a3-4d53-b4b9-7fa7386fc3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:04.206977 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993954c-e8a3-4d53-b4b9-7fa7386fc3b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:04.207608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b993954c-e8a3-4d53-b4b9-7fa7386fc3b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:04.207904 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b993954c-e8a3-4d53-b4b9-7fa7386fc3b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:04.209266 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b993954c-e8a3-4d53-b4b9-7fa7386fc3b1 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:04.221407 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 338/665] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:04.222227 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993954c-e8a3-4d53-b4b9-7fa7386fc3b1 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:04.241687 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:04.251084 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:04.276917 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a607324-b74b-4be2-9f71-ed9a7af26a13 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:04.316173 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7579e5-9c39-4a6f-bf86-19d131ccdc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:04.316725 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7579e5-9c39-4a6f-bf86-19d131ccdc0f None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:04.317005 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7579e5-9c39-4a6f-bf86-19d131ccdc0f None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:04.317256 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7579e5-9c39-4a6f-bf86-19d131ccdc0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:04.327056 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993954c-e8a3-4d53-b4b9-7fa7386fc3b1 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:04.328027 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7579e5-9c39-4a6f-bf86-19d131ccdc0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:04.329470 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7579e5-9c39-4a6f-bf86-19d131ccdc0f None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:04.345115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-2f7579e5-9c39-4a6f-bf86-19d131ccdc0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:04.345115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:04.345115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:04.345115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:04.345115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:04.345115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:04.345115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:04.345115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:04.345115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:04.345115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:04.345115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:04.345115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:04.345115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:04.345115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:04.345115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:04.345115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:04.346538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:04.346538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:04.346538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:04.346538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:04.346538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:04.346538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:04.346538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:04.346538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:04.346538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:04.346538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:04.346538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:04.346538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:04.346538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:04.346538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:04.346538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:04.346538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:04.346538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:04.347416 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:04.347416 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:04.347416 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:04.350907 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 328/666] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:04.351657 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7579e5-9c39-4a6f-bf86-19d131ccdc0f None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:04.365307 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8204c-cdac-4c33-99ce-434e02e020ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:04.365997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8204c-cdac-4c33-99ce-434e02e020ee None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:04.366232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8204c-cdac-4c33-99ce-434e02e020ee None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:04.366493 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8204c-cdac-4c33-99ce-434e02e020ee None admin] PATH_INFO: `/v3/roles/service` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:04.377769 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b8204c-cdac-4c33-99ce-434e02e020ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:20:04.383025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-69b8204c-cdac-4c33-99ce-434e02e020ee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:20:04.383025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:04.383025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:04.383025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:04.383025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:04.383025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:04.383025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:04.383025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:04.383025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:04.383025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:04.383025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:04.383025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:04.383025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:20:04.383025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:20:04.383025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:20:04.383025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:20:04.383969 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:20:04.383969 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:20:04.383969 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:20:04.383969 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:20:04.383969 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:04.383969 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:04.383969 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:04.383969 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:04.383969 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:04.383969 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:04.383969 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:04.383969 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:04.383969 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:04.383969 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:04.383969 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:04.383969 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:04.384805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:04.384805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:04.384805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:04.384805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:04.384805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:20:04.384805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:20:04.384805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:20:04.384805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:20:04.384805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:20:04.384805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:20:04.384805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:20:04.384805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:04.384805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 339/667] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 10:20:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:04.384805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8204c-cdac-4c33-99ce-434e02e020ee None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:04.463475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7579e5-9c39-4a6f-bf86-19d131ccdc0f None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:04.469271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:04.493220 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:04.494200 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8204c-cdac-4c33-99ce-434e02e020ee None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:04.503053 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04221c-5d26-44b6-b525-d9a1fb053886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:04.504228 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04221c-5d26-44b6-b525-d9a1fb053886 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:04.504678 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04221c-5d26-44b6-b525-d9a1fb053886 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:04.505395 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04221c-5d26-44b6-b525-d9a1fb053886 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:04.516703 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc04221c-5d26-44b6-b525-d9a1fb053886 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:04.518368 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc04221c-5d26-44b6-b525-d9a1fb053886 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:04.531902 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cebf60-326e-44ec-b4a9-293c19a0fd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:04.532522 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cebf60-326e-44ec-b4a9-293c19a0fd98 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:04.532795 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cebf60-326e-44ec-b4a9-293c19a0fd98 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:04.533061 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cebf60-326e-44ec-b4a9-293c19a0fd98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:04.534684 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-dc04221c-5d26-44b6-b525-d9a1fb053886 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:04.534684 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:04.534684 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:04.534684 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:04.534684 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:04.534684 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:04.534684 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:04.534684 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:04.534684 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:04.534684 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:04.534684 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:04.534684 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:04.534684 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:04.534684 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:04.534684 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:04.534684 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:04.535802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:04.535802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:04.535802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:04.535802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:04.535802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:04.535802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:04.535802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:04.535802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:04.535802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:04.535802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:04.535802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:04.535802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:04.535802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:04.535802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:04.535802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:04.535802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:04.535802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:04.536910 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:04.536910 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:04.536910 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:04.539008 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 329/668] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:04.539918 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04221c-5d26-44b6-b525-d9a1fb053886 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:04.542202 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cebf60-326e-44ec-b4a9-293c19a0fd98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:04.543631 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cebf60-326e-44ec-b4a9-293c19a0fd98 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:04.555624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-49cebf60-326e-44ec-b4a9-293c19a0fd98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:04.555624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:04.555624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:04.555624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:04.555624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:04.555624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:04.555624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:04.555624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:04.555624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:04.555624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:04.555624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:04.555624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:04.555624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:20:04.555624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:20:04.555624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:20:04.556718 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:20:04.556718 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:04.556718 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:04.556718 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:04.556718 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:04.556718 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:04.556718 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:04.556718 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:04.556718 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:04.556718 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:04.556718 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:04.556718 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:04.556718 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:04.556718 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:04.556718 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:04.556718 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:04.556718 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:04.557614 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:20:04.557614 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:20:04.557614 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:20:04.557614 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:20:04.557614 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:20:04.557614 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:20:04.557614 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:04.557614 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:04.557614 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 340/669] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:20:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:04.558134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cebf60-326e-44ec-b4a9-293c19a0fd98 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:04.657340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04221c-5d26-44b6-b525-d9a1fb053886 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:04.675020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cebf60-326e-44ec-b4a9-293c19a0fd98 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:04.694717 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f8556-e159-4de0-a412-1d99b0c2ec8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:04.695319 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f8556-e159-4de0-a412-1d99b0c2ec8d None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:04.695592 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f8556-e159-4de0-a412-1d99b0c2ec8d None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:04.695858 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f8556-e159-4de0-a412-1d99b0c2ec8d None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:04.696495 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4f8556-e159-4de0-a412-1d99b0c2ec8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:04.696793 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4f8556-e159-4de0-a412-1d99b0c2ec8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:04.698183 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4f8556-e159-4de0-a412-1d99b0c2ec8d None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:04.710358 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 330/670] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:04.711010 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f8556-e159-4de0-a412-1d99b0c2ec8d None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:04.715171 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31589749-7f07-41a9-925a-098478522655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:04.715852 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31589749-7f07-41a9-925a-098478522655 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:04.716156 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31589749-7f07-41a9-925a-098478522655 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:04.716404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31589749-7f07-41a9-925a-098478522655 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:04.717074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31589749-7f07-41a9-925a-098478522655 None admin] RBAC: Adding query filter params (name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:04.717375 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31589749-7f07-41a9-925a-098478522655 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:04.718767 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31589749-7f07-41a9-925a-098478522655 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:04.721285 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:04.729241 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 341/671] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 10:20:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:04.734969 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31589749-7f07-41a9-925a-098478522655 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:04.756032 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:04.819898 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f8556-e159-4de0-a412-1d99b0c2ec8d None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:04.851262 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31589749-7f07-41a9-925a-098478522655 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:04.857508 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3065f6-aa29-4a35-ae69-9a8c3f1528c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:04.858133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3065f6-aa29-4a35-ae69-9a8c3f1528c4 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:04.858407 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3065f6-aa29-4a35-ae69-9a8c3f1528c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:04.858673 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3065f6-aa29-4a35-ae69-9a8c3f1528c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:04.859600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3065f6-aa29-4a35-ae69-9a8c3f1528c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:04.859600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3065f6-aa29-4a35-ae69-9a8c3f1528c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:04.860955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3065f6-aa29-4a35-ae69-9a8c3f1528c4 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:04.872609 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 331/672] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:04.873243 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3065f6-aa29-4a35-ae69-9a8c3f1528c4 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:04.891783 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8682a6-c91f-4c41-ae93-a080e01d8063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:04.892406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8682a6-c91f-4c41-ae93-a080e01d8063 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:04.892690 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8682a6-c91f-4c41-ae93-a080e01d8063 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:04.892961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8682a6-c91f-4c41-ae93-a080e01d8063 None admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:04.893652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8682a6-c91f-4c41-ae93-a080e01d8063 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:04.893957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8682a6-c91f-4c41-ae93-a080e01d8063 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:04.895409 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8682a6-c91f-4c41-ae93-a080e01d8063 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:04.909126 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 342/673] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:20:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:04.914574 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8682a6-c91f-4c41-ae93-a080e01d8063 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:04.985020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:04.986429 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:04.990222 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3065f6-aa29-4a35-ae69-9a8c3f1528c4 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:05.027921 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0c136-7974-4f59-888d-677ec43b4f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:05.028757 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0c136-7974-4f59-888d-677ec43b4f3e None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:05.029044 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0c136-7974-4f59-888d-677ec43b4f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:05.029321 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0c136-7974-4f59-888d-677ec43b4f3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:05.036166 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d0c136-7974-4f59-888d-677ec43b4f3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:05.036285 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8682a6-c91f-4c41-ae93-a080e01d8063 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:05.037631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d0c136-7974-4f59-888d-677ec43b4f3e None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:05.049256 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-82d0c136-7974-4f59-888d-677ec43b4f3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:05.049256 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:05.049256 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:05.049256 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:05.049256 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:05.049256 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:05.049256 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:05.049256 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:05.049256 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:05.049256 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:05.049256 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:05.049256 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:05.049256 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:20:05.049256 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:20:05.049256 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:20:05.050211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:20:05.050211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:05.050211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:05.050211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:05.050211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:05.050211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:05.050211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:05.050211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:05.050211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:05.050211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:05.050211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:05.050211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:05.050211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:05.050211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:05.050211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:05.050211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:05.050211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:05.051136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:20:05.051136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:20:05.051136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:20:05.051136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:20:05.051136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:20:05.051136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:20:05.051136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:05.051136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:05.051136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 332/674] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:20:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:05.052170 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0c136-7974-4f59-888d-677ec43b4f3e None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:05.076747 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076a1d9c-3674-4477-a0ec-a512705aefd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:05.077373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a1d9c-3674-4477-a0ec-a512705aefd8 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:05.077649 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a1d9c-3674-4477-a0ec-a512705aefd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:05.077915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a1d9c-3674-4477-a0ec-a512705aefd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:05.088315 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076a1d9c-3674-4477-a0ec-a512705aefd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:05.089742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076a1d9c-3674-4477-a0ec-a512705aefd8 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:05.106241 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-076a1d9c-3674-4477-a0ec-a512705aefd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:05.106241 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:05.106241 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:05.106241 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:05.106241 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:05.106241 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:05.106241 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:05.106241 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:05.106241 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:05.106241 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:05.106241 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:05.106241 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:05.106241 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:05.106241 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:05.106241 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:05.106241 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:05.107881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:05.107881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:05.107881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:05.107881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:05.107881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:05.107881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:05.107881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:05.107881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:05.107881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:05.107881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:05.107881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:05.107881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:05.107881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:05.107881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:05.107881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:05.107881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:05.107881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:05.108805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:05.108805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:05.108805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:05.108805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 343/675] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:05.108805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076a1d9c-3674-4477-a0ec-a512705aefd8 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:05.158816 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0c136-7974-4f59-888d-677ec43b4f3e None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:05.191750 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a961d5d-8e91-40c8-bfbc-17697799852a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:05.192304 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a961d5d-8e91-40c8-bfbc-17697799852a None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:05.192583 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a961d5d-8e91-40c8-bfbc-17697799852a None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:05.192853 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a961d5d-8e91-40c8-bfbc-17697799852a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:05.199331 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a961d5d-8e91-40c8-bfbc-17697799852a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:05.200790 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a961d5d-8e91-40c8-bfbc-17697799852a None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:05.210105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076a1d9c-3674-4477-a0ec-a512705aefd8 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:05.212050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-1a961d5d-8e91-40c8-bfbc-17697799852a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:05.212050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:05.212050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:05.212050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:05.212050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:05.212050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:05.212050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:05.212050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:05.212050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:05.212050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:05.212050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:05.212050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:05.212050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:20:05.212050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:20:05.212050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:20:05.213072 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:20:05.213072 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:05.213072 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:05.213072 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:05.213072 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:05.213072 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:05.213072 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:05.213072 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:05.213072 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:05.213072 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:05.213072 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:05.213072 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:05.213072 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:05.213072 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:05.213072 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:05.213072 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:05.213072 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:05.214168 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:20:05.214168 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:20:05.214168 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:20:05.214168 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:20:05.214168 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:20:05.214168 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:20:05.214168 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:05.214168 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:05.214168 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 333/676] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:20:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:05.214168 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a961d5d-8e91-40c8-bfbc-17697799852a None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:05.223609 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:05.235563 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:05.249588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137a8cb-ce22-4efb-9388-7d4a9db300a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:05.250233 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137a8cb-ce22-4efb-9388-7d4a9db300a8 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:05.250517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137a8cb-ce22-4efb-9388-7d4a9db300a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:05.250807 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137a8cb-ce22-4efb-9388-7d4a9db300a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:05.252158 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f137a8cb-ce22-4efb-9388-7d4a9db300a8 None admin] RBAC: Adding query filter params (role.id=d36e58344b604bcda1cee49065e8497e, scope.project.id=cff3a6011f4c4893bddffdad3d69f3df, user.id=f841bf31c43b4333a3d834d38b5197e1) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:05.252725 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f137a8cb-ce22-4efb-9388-7d4a9db300a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:05.254839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f137a8cb-ce22-4efb-9388-7d4a9db300a8 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:05.260830 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 344/677] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:20:05 2022] GET /identity/v3/role_assignments?user.id=f841bf31c43b4333a3d834d38b5197e1&scope.project.id=cff3a6011f4c4893bddffdad3d69f3df&role.id=d36e58344b604bcda1cee49065e8497e => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:05.262114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e08196-4ef0-4534-9fbe-24d775e18ba2 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:05.262405 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e08196-4ef0-4534-9fbe-24d775e18ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:05.262684 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e08196-4ef0-4534-9fbe-24d775e18ba2 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:05.268885 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 345/678] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:05.268885 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e08196-4ef0-4534-9fbe-24d775e18ba2 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:05.332653 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a961d5d-8e91-40c8-bfbc-17697799852a None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:05.369585 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d14c2-7c0b-488f-81d2-092a533eafdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:05.370350 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d14c2-7c0b-488f-81d2-092a533eafdc None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:05.371127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d14c2-7c0b-488f-81d2-092a533eafdc None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:05.371409 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d14c2-7c0b-488f-81d2-092a533eafdc None admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:05.372122 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d14c2-7c0b-488f-81d2-092a533eafdc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:05.373005 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d14c2-7c0b-488f-81d2-092a533eafdc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:05.374500 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d14c2-7c0b-488f-81d2-092a533eafdc None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:05.387385 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 334/679] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:20:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:05.388334 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d14c2-7c0b-488f-81d2-092a533eafdc None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:05.389385 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e08196-4ef0-4534-9fbe-24d775e18ba2 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:05.430368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c950b0-c6f9-490e-ba4d-28b5cce5125e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:05.430972 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c950b0-c6f9-490e-ba4d-28b5cce5125e None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:05.431250 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c950b0-c6f9-490e-ba4d-28b5cce5125e None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:05.431521 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c950b0-c6f9-490e-ba4d-28b5cce5125e None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:05.432165 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c950b0-c6f9-490e-ba4d-28b5cce5125e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:05.432466 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c950b0-c6f9-490e-ba4d-28b5cce5125e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:05.433829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c950b0-c6f9-490e-ba4d-28b5cce5125e None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:05.447088 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 346/680] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:05.448107 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14ef21-3609-4eff-a03b-979d15b65ebd None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:05.448403 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14ef21-3609-4eff-a03b-979d15b65ebd None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:05.448674 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14ef21-3609-4eff-a03b-979d15b65ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:05.469054 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-bc14ef21-3609-4eff-a03b-979d15b65ebd None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:05.470287 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-bc14ef21-3609-4eff-a03b-979d15b65ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:05.484684 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:05.503681 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:05.507247 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d14c2-7c0b-488f-81d2-092a533eafdc None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:05.544685 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6790b4bc-a6da-4109-97c4-4057717642a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:05.545285 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6790b4bc-a6da-4109-97c4-4057717642a5 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:05.545559 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6790b4bc-a6da-4109-97c4-4057717642a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:05.545828 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6790b4bc-a6da-4109-97c4-4057717642a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:05.546573 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6790b4bc-a6da-4109-97c4-4057717642a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:05.546877 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6790b4bc-a6da-4109-97c4-4057717642a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:05.548293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6790b4bc-a6da-4109-97c4-4057717642a5 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:05.560918 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 347/681] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:20:05.560918 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14ef21-3609-4eff-a03b-979d15b65ebd None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:05.564908 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 335/682] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:20:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:05.565799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6790b4bc-a6da-4109-97c4-4057717642a5 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:05.672664 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6790b4bc-a6da-4109-97c4-4057717642a5 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:05.682506 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14ef21-3609-4eff-a03b-979d15b65ebd None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:05.708375 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229392a-d376-4aa3-88a8-35efdcf02be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:05.708926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229392a-d376-4aa3-88a8-35efdcf02be7 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:05.709202 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229392a-d376-4aa3-88a8-35efdcf02be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:05.709469 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229392a-d376-4aa3-88a8-35efdcf02be7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:05.717464 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0229392a-d376-4aa3-88a8-35efdcf02be7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:05.718953 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0229392a-d376-4aa3-88a8-35efdcf02be7 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:05.721593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:05.724843 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9682b1d-25e3-4c6d-823f-33dbae25871a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:05.725573 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9682b1d-25e3-4c6d-823f-33dbae25871a None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:05.725956 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9682b1d-25e3-4c6d-823f-33dbae25871a None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:05.726388 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9682b1d-25e3-4c6d-823f-33dbae25871a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:05.727216 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9682b1d-25e3-4c6d-823f-33dbae25871a None admin] RBAC: Adding query filter params (role.id=d36e58344b604bcda1cee49065e8497e, scope.project.id=cff3a6011f4c4893bddffdad3d69f3df, user.id=41b242fbe2504f8cac26ce4ff46f5390) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:05.727659 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9682b1d-25e3-4c6d-823f-33dbae25871a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:05.729267 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9682b1d-25e3-4c6d-823f-33dbae25871a None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:05.731836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-0229392a-d376-4aa3-88a8-35efdcf02be7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 26 10:20:05.731836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:05.731836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:05.731836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:05.731836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:05.731836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:05.731836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:05.731836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:05.731836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:05.731836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:05.731836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:05.731836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:05.731836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:20:05.731836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:20:05.731836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:20:05.731836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:20:05.732933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:05.732933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:05.732933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:20:05.732933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:05.732933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:20:05.732933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:05.732933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:05.732933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:05.732933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:05.732933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:05.732933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:05.732933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:05.732933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:05.732933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:05.732933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:05.732933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:05.732933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:05.732933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:05.734121 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:05.734121 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:05.734121 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:20:05.734121 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:20:05.734121 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:20:05.734121 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:20:05.734121 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:20:05.734121 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:20:05.734121 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 26 10:20:05.734121 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:05.736233 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 336/683] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:20:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:05.736593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229392a-d376-4aa3-88a8-35efdcf02be7 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:05.738399 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 348/684] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:20:05 2022] GET /identity/v3/role_assignments?user.id=41b242fbe2504f8cac26ce4ff46f5390&scope.project.id=cff3a6011f4c4893bddffdad3d69f3df&role.id=d36e58344b604bcda1cee49065e8497e => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:05.740787 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9682b1d-25e3-4c6d-823f-33dbae25871a None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:05.777490 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:05.872177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9682b1d-25e3-4c6d-823f-33dbae25871a None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:05.914412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0a60c-38d2-45b5-a3af-65e4a78f7f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:05.915063 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0a60c-38d2-45b5-a3af-65e4a78f7f0a None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:05.915342 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0a60c-38d2-45b5-a3af-65e4a78f7f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:05.915616 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0a60c-38d2-45b5-a3af-65e4a78f7f0a None admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:05.916550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0a60c-38d2-45b5-a3af-65e4a78f7f0a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:05.917209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0a60c-38d2-45b5-a3af-65e4a78f7f0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:05.918969 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0a60c-38d2-45b5-a3af-65e4a78f7f0a None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:05.919908 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229392a-d376-4aa3-88a8-35efdcf02be7 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:05.943393 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 349/685] 10.222.0.49 () {56 vars in 1135 bytes} [Mon Sep 26 10:20:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:05.944145 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0a60c-38d2-45b5-a3af-65e4a78f7f0a None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:05.970288 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a717101-9c66-44ce-9945-dcac420ecc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:05.970828 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a717101-9c66-44ce-9945-dcac420ecc0c None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:05.971110 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a717101-9c66-44ce-9945-dcac420ecc0c None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:05.971381 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a717101-9c66-44ce-9945-dcac420ecc0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:05.972116 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a717101-9c66-44ce-9945-dcac420ecc0c None admin] RBAC: Adding query filter params (role.id=d36e58344b604bcda1cee49065e8497e, scope.project.id=cff3a6011f4c4893bddffdad3d69f3df, user.id=feddae5ba018426697854931a55104e3) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:05.972423 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a717101-9c66-44ce-9945-dcac420ecc0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:05.974131 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a717101-9c66-44ce-9945-dcac420ecc0c None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:05.981159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 337/686] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:20:05 2022] GET /identity/v3/role_assignments?user.id=feddae5ba018426697854931a55104e3&scope.project.id=cff3a6011f4c4893bddffdad3d69f3df&role.id=d36e58344b604bcda1cee49065e8497e => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:05.982041 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a717101-9c66-44ce-9945-dcac420ecc0c None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:06.069912 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:06.080302 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:06.093943 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0a60c-38d2-45b5-a3af-65e4a78f7f0a None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:06.145898 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f078ef-5474-44dd-b381-807d8ee1bcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:06.146661 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f078ef-5474-44dd-b381-807d8ee1bcc0 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:06.146942 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f078ef-5474-44dd-b381-807d8ee1bcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:06.147212 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f078ef-5474-44dd-b381-807d8ee1bcc0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:06.149595 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a717101-9c66-44ce-9945-dcac420ecc0c None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:06.166192 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f078ef-5474-44dd-b381-807d8ee1bcc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:20:06.173744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-19f078ef-5474-44dd-b381-807d8ee1bcc0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:20:06.173744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:06.173744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:06.173744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:06.173744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:06.173744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:06.173744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:06.173744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:06.173744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:06.173744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:06.173744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:06.173744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:06.173744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:20:06.173744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:20:06.173744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:20:06.173744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:20:06.174856 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:20:06.174856 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:20:06.174856 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:20:06.174856 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:20:06.174856 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:06.174856 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:06.174856 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:06.174856 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:06.174856 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:06.174856 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:06.174856 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:06.174856 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:06.174856 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:06.174856 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:06.174856 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:06.174856 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:06.175705 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:06.175705 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:06.175705 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:06.175705 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:06.175705 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:20:06.175705 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:20:06.175705 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:20:06.175705 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:20:06.175705 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:20:06.175705 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:20:06.175705 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:20:06.175705 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:06.175705 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 350/687] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:20:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:06.183595 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f078ef-5474-44dd-b381-807d8ee1bcc0 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:06.193925 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780314d-a106-40d5-bb36-90c82bb8c37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:06.194736 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780314d-a106-40d5-bb36-90c82bb8c37d None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:06.195017 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780314d-a106-40d5-bb36-90c82bb8c37d None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:06.195326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780314d-a106-40d5-bb36-90c82bb8c37d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:06.285420 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780314d-a106-40d5-bb36-90c82bb8c37d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:06.288244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780314d-a106-40d5-bb36-90c82bb8c37d None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:06.321829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f078ef-5474-44dd-b381-807d8ee1bcc0 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:06.366751 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac2aa1-3335-4c56-9b7d-d86f43d64cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:06.367747 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac2aa1-3335-4c56-9b7d-d86f43d64cec None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:06.368250 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac2aa1-3335-4c56-9b7d-d86f43d64cec None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:06.368761 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac2aa1-3335-4c56-9b7d-d86f43d64cec None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:06.369453 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:06.370890 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac2aa1-3335-4c56-9b7d-d86f43d64cec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:06.371444 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac2aa1-3335-4c56-9b7d-d86f43d64cec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:06.375343 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac2aa1-3335-4c56-9b7d-d86f43d64cec None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:06.386223 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 351/688] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:20:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:06.392600 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac2aa1-3335-4c56-9b7d-d86f43d64cec None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:06.409343 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:06.426095 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 338/689] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:20:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 10:20:06.435596 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780314d-a106-40d5-bb36-90c82bb8c37d None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:06.519057 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac2aa1-3335-4c56-9b7d-d86f43d64cec None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:06.556620 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14467a19-57cd-46ec-a4ad-42ae925d14f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:06.557588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14467a19-57cd-46ec-a4ad-42ae925d14f1 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:06.558101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14467a19-57cd-46ec-a4ad-42ae925d14f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:06.558582 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14467a19-57cd-46ec-a4ad-42ae925d14f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:06.577144 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14467a19-57cd-46ec-a4ad-42ae925d14f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:06.579290 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14467a19-57cd-46ec-a4ad-42ae925d14f1 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:06.600665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-14467a19-57cd-46ec-a4ad-42ae925d14f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:06.600665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:06.600665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:06.600665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:06.600665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:06.600665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:06.600665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:06.600665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:06.600665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:06.600665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:06.600665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:06.600665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:06.600665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:06.600665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:06.600665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:06.600665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:06.601652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:06.601652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:06.601652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:06.601652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:06.601652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:06.601652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:06.601652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:06.601652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:06.601652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:06.601652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:06.601652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:06.601652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:06.601652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:06.601652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:06.601652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:06.601652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:06.601652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:06.602625 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:06.602625 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:06.602625 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:06.604770 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 352/690] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:06.610791 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14467a19-57cd-46ec-a4ad-42ae925d14f1 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:06.615616 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780314d-a106-40d5-bb36-90c82bb8c37d None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:06.689063 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d13fa0-605f-4112-8dea-fc848a92e555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:06.693004 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d13fa0-605f-4112-8dea-fc848a92e555 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:06.693539 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d13fa0-605f-4112-8dea-fc848a92e555 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:06.694005 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d13fa0-605f-4112-8dea-fc848a92e555 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:06.715397 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d13fa0-605f-4112-8dea-fc848a92e555 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:06.722456 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d13fa0-605f-4112-8dea-fc848a92e555 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:06.727991 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:06.746118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-75d13fa0-605f-4112-8dea-fc848a92e555 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:06.746118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:06.746118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:06.746118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:06.746118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:06.746118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:06.746118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:06.746118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:06.746118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:06.746118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:06.746118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:06.746118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:06.746118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:06.746118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:06.746118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:06.746118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:06.747123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:06.747123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:06.747123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:06.747123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:06.747123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:06.747123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:06.747123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:06.747123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:06.747123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:06.747123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:06.747123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:06.747123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:06.747123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:06.747123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:06.747123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:06.747123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:06.747123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:06.748127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:06.748127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:06.748127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:06.754506 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 339/691] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:06.759563 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d13fa0-605f-4112-8dea-fc848a92e555 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:06.779379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14467a19-57cd-46ec-a4ad-42ae925d14f1 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:06.797989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:06.818974 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e72c0c-1af1-4f42-99ce-a745eb86600a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:06.820004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e72c0c-1af1-4f42-99ce-a745eb86600a None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:06.820535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e72c0c-1af1-4f42-99ce-a745eb86600a None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:06.820997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e72c0c-1af1-4f42-99ce-a745eb86600a None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:06.821835 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e72c0c-1af1-4f42-99ce-a745eb86600a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:06.822357 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e72c0c-1af1-4f42-99ce-a745eb86600a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:06.823990 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e72c0c-1af1-4f42-99ce-a745eb86600a None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:06.838380 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 353/692] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:06.849221 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e72c0c-1af1-4f42-99ce-a745eb86600a None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:06.952883 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e72c0c-1af1-4f42-99ce-a745eb86600a None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:06.975940 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:06.983452 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d13fa0-605f-4112-8dea-fc848a92e555 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:07.004190 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fc869-3a83-4dcd-8f2b-2442ddba109d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:07.004838 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fc869-3a83-4dcd-8f2b-2442ddba109d None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:07.005115 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fc869-3a83-4dcd-8f2b-2442ddba109d None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:07.005381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fc869-3a83-4dcd-8f2b-2442ddba109d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:07.023139 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7fc869-3a83-4dcd-8f2b-2442ddba109d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:07.024566 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7fc869-3a83-4dcd-8f2b-2442ddba109d None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:07.024821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b321b4f-5ef0-494b-a0ea-d692e5dea911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:07.025706 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b321b4f-5ef0-494b-a0ea-d692e5dea911 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:07.026221 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b321b4f-5ef0-494b-a0ea-d692e5dea911 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:07.026709 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b321b4f-5ef0-494b-a0ea-d692e5dea911 None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:07.027546 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b321b4f-5ef0-494b-a0ea-d692e5dea911 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:07.028037 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b321b4f-5ef0-494b-a0ea-d692e5dea911 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:07.030352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b321b4f-5ef0-494b-a0ea-d692e5dea911 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:07.043663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 340/693] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:07.048829 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b321b4f-5ef0-494b-a0ea-d692e5dea911 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:07.051475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-2f7fc869-3a83-4dcd-8f2b-2442ddba109d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 10:20:07.051475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:07.051475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:07.051475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:07.051475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:07.051475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:07.051475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:07.051475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:07.051475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:07.051475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:07.051475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:07.051475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:07.051475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:20:07.051475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:20:07.051475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:20:07.051475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:20:07.052421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:07.052421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:07.052421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:20:07.052421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:07.052421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:20:07.052421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:07.052421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:07.052421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:07.052421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:07.052421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:07.052421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:07.052421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:07.052421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:07.052421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:07.052421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:07.052421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:07.052421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:07.052421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:07.053371 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:07.053371 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:07.053371 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:20:07.053371 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:20:07.053371 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:20:07.053371 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:20:07.053371 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:20:07.053371 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:20:07.053371 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 10:20:07.053371 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:07.053371 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 354/694] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Sep 26 10:20:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:20:07.059023 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fc869-3a83-4dcd-8f2b-2442ddba109d None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:07.185929 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fc869-3a83-4dcd-8f2b-2442ddba109d None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:07.208314 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:07.230046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70417c7f-395c-4ef9-9349-028b9df81ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:07.230672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70417c7f-395c-4ef9-9349-028b9df81ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:07.230962 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70417c7f-395c-4ef9-9349-028b9df81ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:07.231219 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70417c7f-395c-4ef9-9349-028b9df81ec1 None admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:07.231989 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70417c7f-395c-4ef9-9349-028b9df81ec1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:07.232294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70417c7f-395c-4ef9-9349-028b9df81ec1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:07.233717 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70417c7f-395c-4ef9-9349-028b9df81ec1 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:07.266673 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b321b4f-5ef0-494b-a0ea-d692e5dea911 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:07.279036 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 355/695] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Sep 26 10:20:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:07.281500 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70417c7f-395c-4ef9-9349-028b9df81ec1 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:07.339524 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:07.347071 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b400f-2d3b-4423-ade9-ebeddf576ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:07.347725 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b400f-2d3b-4423-ade9-ebeddf576ddc None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:07.348047 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b400f-2d3b-4423-ade9-ebeddf576ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:07.348253 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b400f-2d3b-4423-ade9-ebeddf576ddc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:07.362873 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b400f-2d3b-4423-ade9-ebeddf576ddc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:07.364364 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b400f-2d3b-4423-ade9-ebeddf576ddc None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:07.384190 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-d90b400f-2d3b-4423-ade9-ebeddf576ddc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:07.384190 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:07.384190 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:07.384190 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:07.384190 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:07.384190 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:07.384190 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:07.384190 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:07.384190 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:07.384190 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:07.384190 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:07.384190 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:07.384190 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:20:07.384190 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:20:07.384190 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:20:07.385176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:20:07.385176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:07.385176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:07.385176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:07.385176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:07.385176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:07.385176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:07.385176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:07.385176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:07.385176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:07.385176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:07.385176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:07.385176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:07.385176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:07.385176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:07.385176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:07.385176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:07.386134 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:20:07.386134 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:20:07.386134 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:20:07.386134 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:20:07.386134 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:20:07.386134 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:20:07.386134 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:07.386134 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:07.386134 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 341/696] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:20:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:20:07.391251 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b400f-2d3b-4423-ade9-ebeddf576ddc None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:07.412066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70417c7f-395c-4ef9-9349-028b9df81ec1 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:07.448614 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c51006-6f72-439b-8d8c-54ebfab102b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:07.449559 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c51006-6f72-439b-8d8c-54ebfab102b2 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:07.450053 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c51006-6f72-439b-8d8c-54ebfab102b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:07.450576 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c51006-6f72-439b-8d8c-54ebfab102b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:07.523606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c51006-6f72-439b-8d8c-54ebfab102b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:07.525415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c51006-6f72-439b-8d8c-54ebfab102b2 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:07.566978 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:07.567862 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b400f-2d3b-4423-ade9-ebeddf576ddc None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:07.640412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 356/697] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:20:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 10:20:07.647693 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c51006-6f72-439b-8d8c-54ebfab102b2 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:07.663994 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8119a8-7afa-49b3-8abe-298d3c6f4f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:07.664613 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8119a8-7afa-49b3-8abe-298d3c6f4f70 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:07.664891 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8119a8-7afa-49b3-8abe-298d3c6f4f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:07.665159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8119a8-7afa-49b3-8abe-298d3c6f4f70 None admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:07.665862 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8119a8-7afa-49b3-8abe-298d3c6f4f70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:07.666195 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8119a8-7afa-49b3-8abe-298d3c6f4f70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:07.667620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8119a8-7afa-49b3-8abe-298d3c6f4f70 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:07.690394 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:07.694861 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 342/698] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:20:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:07.698857 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8119a8-7afa-49b3-8abe-298d3c6f4f70 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:07.797893 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c51006-6f72-439b-8d8c-54ebfab102b2 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:07.819187 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8119a8-7afa-49b3-8abe-298d3c6f4f70 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:07.847335 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e1f7ee-cc4a-459f-897b-041349b0cd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:07.848076 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e1f7ee-cc4a-459f-897b-041349b0cd21 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:07.848209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e1f7ee-cc4a-459f-897b-041349b0cd21 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:07.848480 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e1f7ee-cc4a-459f-897b-041349b0cd21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:07.856022 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:07.860588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e1f7ee-cc4a-459f-897b-041349b0cd21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:07.862992 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e1f7ee-cc4a-459f-897b-041349b0cd21 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:07.866043 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2fa9ba-c429-4755-8933-9cd00f255170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:07.867685 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fa9ba-c429-4755-8933-9cd00f255170 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:07.868307 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fa9ba-c429-4755-8933-9cd00f255170 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:07.868795 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fa9ba-c429-4755-8933-9cd00f255170 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:07.869721 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2fa9ba-c429-4755-8933-9cd00f255170 None admin] RBAC: Adding query filter params (role.id=d36e58344b604bcda1cee49065e8497e, scope.project.id=cff3a6011f4c4893bddffdad3d69f3df, user.id=5e6be34fc93a4551a83e5b8380188a6b) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:07.870797 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2fa9ba-c429-4755-8933-9cd00f255170 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:07.873502 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2fa9ba-c429-4755-8933-9cd00f255170 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:07.878980 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-39e1f7ee-cc4a-459f-897b-041349b0cd21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:07.878980 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:07.878980 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:07.878980 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:07.878980 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:07.878980 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:07.878980 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:07.878980 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:07.878980 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:07.878980 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:07.878980 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:07.878980 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:07.878980 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:07.878980 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:07.878980 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:07.878980 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:07.879930 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:07.879930 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:07.879930 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:07.879930 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:07.879930 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:07.879930 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:07.879930 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:07.879930 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:07.879930 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:07.879930 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:07.879930 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:07.879930 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:07.879930 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:07.879930 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:07.879930 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:07.879930 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:07.879930 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:07.880852 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:07.880852 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:07.880852 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:07.883213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 343/700] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:20:07 2022] GET /identity/v3/role_assignments?user.id=5e6be34fc93a4551a83e5b8380188a6b&scope.project.id=cff3a6011f4c4893bddffdad3d69f3df&role.id=d36e58344b604bcda1cee49065e8497e => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:07.884795 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 357/700] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:07.885190 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72837ec4-8c61-4cb6-9fb4-ab98729f8b71 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:07.885625 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e1f7ee-cc4a-459f-897b-041349b0cd21 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:07.885878 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72837ec4-8c61-4cb6-9fb4-ab98729f8b71 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:07.886553 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72837ec4-8c61-4cb6-9fb4-ab98729f8b71 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:07.889239 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 344/701] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:07.900021 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae15ecc-d3ba-45d2-9a62-4b36d33167ca None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:07.900021 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae15ecc-d3ba-45d2-9a62-4b36d33167ca None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:07.900021 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae15ecc-d3ba-45d2-9a62-4b36d33167ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:07.918615 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-5ae15ecc-d3ba-45d2-9a62-4b36d33167ca None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:07.919901 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-5ae15ecc-d3ba-45d2-9a62-4b36d33167ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:08.040553 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:08.069137 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 345/702] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:20:08.095084 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e1f7ee-cc4a-459f-897b-041349b0cd21 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:08.134620 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6fb2d1-589d-44e7-99e8-18f9019fc9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:08.135228 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fb2d1-589d-44e7-99e8-18f9019fc9ac None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:08.135502 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fb2d1-589d-44e7-99e8-18f9019fc9ac None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:08.135774 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fb2d1-589d-44e7-99e8-18f9019fc9ac None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:08.136415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6fb2d1-589d-44e7-99e8-18f9019fc9ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:08.136712 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6fb2d1-589d-44e7-99e8-18f9019fc9ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:08.138809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6fb2d1-589d-44e7-99e8-18f9019fc9ac None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:08.152056 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 358/703] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:08.157520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae15ecc-d3ba-45d2-9a62-4b36d33167ca None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:08.279868 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae15ecc-d3ba-45d2-9a62-4b36d33167ca None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:08.319658 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb711a-9e4a-44e8-93dd-c8fc66ce8f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:08.320287 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb711a-9e4a-44e8-93dd-c8fc66ce8f47 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:08.320560 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb711a-9e4a-44e8-93dd-c8fc66ce8f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:08.320825 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb711a-9e4a-44e8-93dd-c8fc66ce8f47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:08.328299 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb711a-9e4a-44e8-93dd-c8fc66ce8f47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:08.329726 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb711a-9e4a-44e8-93dd-c8fc66ce8f47 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:08.341593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-01fb711a-9e4a-44e8-93dd-c8fc66ce8f47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:08.341593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:08.341593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:08.341593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:08.341593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:08.341593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:08.341593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:08.341593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:08.341593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:08.341593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:08.341593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:08.341593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:08.341593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:20:08.341593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:20:08.341593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:20:08.342521 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:20:08.342521 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:08.342521 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:08.342521 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:08.342521 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:08.342521 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:08.342521 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:08.342521 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:08.342521 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:08.342521 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:08.342521 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:08.342521 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:08.342521 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:08.342521 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:08.342521 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:08.342521 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:08.342521 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:08.343964 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:20:08.343964 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:20:08.343964 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:20:08.343964 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:20:08.343964 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:20:08.343964 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:20:08.343964 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:08.343964 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:08.343964 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 346/704] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:20:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:08.348406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6fb2d1-589d-44e7-99e8-18f9019fc9ac None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:08.387697 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb711a-9e4a-44e8-93dd-c8fc66ce8f47 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:08.417247 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:08.470430 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:08.482935 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6fb2d1-589d-44e7-99e8-18f9019fc9ac None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:08.521644 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412ea41-28ab-4786-8b27-76bcd38f0603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:08.522312 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412ea41-28ab-4786-8b27-76bcd38f0603 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:08.522651 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412ea41-28ab-4786-8b27-76bcd38f0603 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:08.522931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412ea41-28ab-4786-8b27-76bcd38f0603 None admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:08.523634 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d412ea41-28ab-4786-8b27-76bcd38f0603 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:08.523937 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d412ea41-28ab-4786-8b27-76bcd38f0603 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:08.525369 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d412ea41-28ab-4786-8b27-76bcd38f0603 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:08.539979 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 359/705] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:20:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:08.546184 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412ea41-28ab-4786-8b27-76bcd38f0603 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:08.564246 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb711a-9e4a-44e8-93dd-c8fc66ce8f47 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:08.659149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7eb352-a5c3-40f0-87d7-ad48ba18eada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:08.659780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7eb352-a5c3-40f0-87d7-ad48ba18eada None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:08.660065 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7eb352-a5c3-40f0-87d7-ad48ba18eada None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:08.660336 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7eb352-a5c3-40f0-87d7-ad48ba18eada None admin] PATH_INFO: `/v3/services/network` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:08.660437 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412ea41-28ab-4786-8b27-76bcd38f0603 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:08.660932 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7eb352-a5c3-40f0-87d7-ad48ba18eada None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:08.662508 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7eb352-a5c3-40f0-87d7-ad48ba18eada None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:08.674261 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:08.683179 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-9d7eb352-a5c3-40f0-87d7-ad48ba18eada None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 10:20:08.683179 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:08.683179 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 10:20:08.683179 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 10:20:08.683179 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 10:20:08.683179 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 10:20:08.683179 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 10:20:08.683179 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:08.683179 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 10:20:08.683179 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:08.683179 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 10:20:08.683179 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:08.683179 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:08.683179 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:08.683179 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:08.683179 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:08.683179 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:08.684189 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:08.684189 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:08.684189 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:08.684189 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:08.684189 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:08.684189 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:08.684189 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 10:20:08.684189 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 10:20:08.684189 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 10:20:08.684189 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 10:20:08.684189 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:08.684189 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:08.684189 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:08.684189 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:08.684189 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:08.684189 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:08.685102 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:08.685102 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:08.685102 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:08.685102 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:08.685102 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:08.685102 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:08.685102 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:08.685102 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:08.685102 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:08.685102 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:08.685102 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 10:20:08.685102 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:08.685102 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 10:20:08.685102 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:08.687815 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 347/706] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:20:08 2022] GET /identity/v3/services/network => generated 88 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:20:08.688982 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8780332-a3e5-4455-baa6-d736a5d791e7 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:08.689257 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8780332-a3e5-4455-baa6-d736a5d791e7 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:08.689526 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8780332-a3e5-4455-baa6-d736a5d791e7 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:08.699852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 348/707] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:08.699852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8780332-a3e5-4455-baa6-d736a5d791e7 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:08.708946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e2fa15-e582-4646-881f-36aaa08e24e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:08.710327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2fa15-e582-4646-881f-36aaa08e24e4 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:08.710837 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2fa15-e582-4646-881f-36aaa08e24e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:08.711713 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2fa15-e582-4646-881f-36aaa08e24e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:08.712958 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e2fa15-e582-4646-881f-36aaa08e24e4 None admin] RBAC: Adding query filter params (role.id=532b37d9076c45618cf352aafb85f35e, scope.project.id=cff3a6011f4c4893bddffdad3d69f3df, user.id=cee41014d5d043888c6b399710bccab4) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:08.713491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e2fa15-e582-4646-881f-36aaa08e24e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:08.716521 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e2fa15-e582-4646-881f-36aaa08e24e4 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:08.724779 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 360/708] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:20:08 2022] GET /identity/v3/role_assignments?user.id=cee41014d5d043888c6b399710bccab4&scope.project.id=cff3a6011f4c4893bddffdad3d69f3df&role.id=532b37d9076c45618cf352aafb85f35e => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:08.726398 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbb0d4-4007-4df3-8ffe-4be4fa1499a2 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:08.726888 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbb0d4-4007-4df3-8ffe-4be4fa1499a2 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:08.727399 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbb0d4-4007-4df3-8ffe-4be4fa1499a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:08.751714 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-73dbb0d4-4007-4df3-8ffe-4be4fa1499a2 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:08.754381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-73dbb0d4-4007-4df3-8ffe-4be4fa1499a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:08.846315 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8780332-a3e5-4455-baa6-d736a5d791e7 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:08.854075 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 361/709] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:20:08.856004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a6722-73f2-4425-87ff-63dbd3798488 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:08.856359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a6722-73f2-4425-87ff-63dbd3798488 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:08.856774 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a6722-73f2-4425-87ff-63dbd3798488 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:08.862093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 362/710] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:08.864263 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434f741-af8d-4b25-9074-d24b3cae17eb None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:08.864727 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434f741-af8d-4b25-9074-d24b3cae17eb None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:08.865188 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434f741-af8d-4b25-9074-d24b3cae17eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:08.879359 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:08.885341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-9434f741-af8d-4b25-9074-d24b3cae17eb None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:08.887038 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-9434f741-af8d-4b25-9074-d24b3cae17eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:08.908941 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0fdd8-6eca-47b8-8d77-0987ac56b766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:08.909712 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0fdd8-6eca-47b8-8d77-0987ac56b766 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:08.910105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0fdd8-6eca-47b8-8d77-0987ac56b766 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:08.910385 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0fdd8-6eca-47b8-8d77-0987ac56b766 None admin] PATH_INFO: `/v3/services` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:08.911099 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0fdd8-6eca-47b8-8d77-0987ac56b766 None admin] RBAC: Adding query filter params (name=network) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:08.911406 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0fdd8-6eca-47b8-8d77-0987ac56b766 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:08.912946 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0fdd8-6eca-47b8-8d77-0987ac56b766 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:08.926229 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 349/711] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Sep 26 10:20:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:08.935308 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0fdd8-6eca-47b8-8d77-0987ac56b766 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:08.977979 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 363/712] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:20:09.073033 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0fdd8-6eca-47b8-8d77-0987ac56b766 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:09.124195 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0da3b-ee43-4aa8-b226-56c746f43b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:09.125211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0da3b-ee43-4aa8-b226-56c746f43b2f None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:09.125764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0da3b-ee43-4aa8-b226-56c746f43b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:09.126314 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0da3b-ee43-4aa8-b226-56c746f43b2f None admin] PATH_INFO: `/v3/services` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:09.127156 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c0da3b-ee43-4aa8-b226-56c746f43b2f None admin] RBAC: Adding query filter params (type=network) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:09.127660 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c0da3b-ee43-4aa8-b226-56c746f43b2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:09.129277 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c0da3b-ee43-4aa8-b226-56c746f43b2f None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:09.143095 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 350/713] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Sep 26 10:20:08 2022] GET /identity/v3/services?type=network => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:09.188021 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9434f741-af8d-4b25-9074-d24b3cae17eb None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:09.232625 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:09.301150 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0da3b-ee43-4aa8-b226-56c746f43b2f None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:09.354547 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9434f741-af8d-4b25-9074-d24b3cae17eb None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:09.375074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:09.407241 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fcfa34-c967-4a1c-800d-e4ed2e062ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:09.408542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fcfa34-c967-4a1c-800d-e4ed2e062ebe None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:09.409031 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fcfa34-c967-4a1c-800d-e4ed2e062ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:09.410170 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fcfa34-c967-4a1c-800d-e4ed2e062ebe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:09.424567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fcfa34-c967-4a1c-800d-e4ed2e062ebe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:09.427049 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fcfa34-c967-4a1c-800d-e4ed2e062ebe None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:09.446437 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-10fcfa34-c967-4a1c-800d-e4ed2e062ebe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:09.446437 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:09.446437 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:09.446437 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:09.446437 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:09.446437 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:09.446437 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:09.446437 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:09.446437 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:09.446437 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:09.446437 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:09.446437 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:09.446437 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:09.446437 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:09.446437 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:09.446437 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:09.447529 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:09.447529 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:09.447529 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:09.447529 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:09.447529 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:09.447529 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:09.447529 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:09.447529 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:09.447529 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:09.447529 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:09.447529 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:09.447529 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:09.447529 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:09.447529 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:09.447529 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:09.447529 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:09.447529 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:09.448420 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:09.448420 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:09.448420 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:09.450736 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 364/714] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:09.458044 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0da3b-ee43-4aa8-b226-56c746f43b2f None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:09.458968 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fcfa34-c967-4a1c-800d-e4ed2e062ebe None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:09.516235 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fc561-211a-4180-92ac-00014aa7093a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:09.516863 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fc561-211a-4180-92ac-00014aa7093a None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:09.517137 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fc561-211a-4180-92ac-00014aa7093a None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:09.517403 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fc561-211a-4180-92ac-00014aa7093a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:09.517982 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3fc561-211a-4180-92ac-00014aa7093a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:09.519638 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3fc561-211a-4180-92ac-00014aa7093a None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:09.538909 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-3b3fc561-211a-4180-92ac-00014aa7093a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 10:20:09.538909 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:09.538909 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 10:20:09.538909 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 10:20:09.538909 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 10:20:09.538909 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 10:20:09.538909 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 10:20:09.538909 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:09.538909 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 10:20:09.538909 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:09.538909 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 10:20:09.538909 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:09.538909 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:09.538909 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:09.538909 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:09.538909 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:09.538909 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:09.539969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:09.539969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:09.539969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:09.539969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:09.539969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:09.539969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:09.539969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 10:20:09.539969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 10:20:09.539969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 10:20:09.539969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 10:20:09.539969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:09.539969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:09.539969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:09.539969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:09.539969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:09.539969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:09.540821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:09.540821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:09.540821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:09.540821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:09.540821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:09.540821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:09.540821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:09.540821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:09.540821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:09.540821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:09.540821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 10:20:09.540821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:09.540821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 10:20:09.540821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:09.540821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 351/715] 10.222.0.49 () {56 vars in 1129 bytes} [Mon Sep 26 10:20:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:09.549377 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fc561-211a-4180-92ac-00014aa7093a None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:09.622975 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:09.642678 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fcfa34-c967-4a1c-800d-e4ed2e062ebe None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:09.703825 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5f469-a3d1-4c6f-b463-b8a9ae064e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:09.704352 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5f469-a3d1-4c6f-b463-b8a9ae064e29 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:09.704629 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5f469-a3d1-4c6f-b463-b8a9ae064e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:09.704897 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5f469-a3d1-4c6f-b463-b8a9ae064e29 None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:09.705537 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5f469-a3d1-4c6f-b463-b8a9ae064e29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:09.705834 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5f469-a3d1-4c6f-b463-b8a9ae064e29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:09.707499 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5f469-a3d1-4c6f-b463-b8a9ae064e29 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:09.729408 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 365/716] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:09.735356 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5f469-a3d1-4c6f-b463-b8a9ae064e29 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:09.737341 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fc561-211a-4180-92ac-00014aa7093a None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:09.744700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:09.776538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a356a-275e-4dba-b854-7330949f6cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:09.777502 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a356a-275e-4dba-b854-7330949f6cdd None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:09.777977 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a356a-275e-4dba-b854-7330949f6cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:09.778501 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a356a-275e-4dba-b854-7330949f6cdd None admin] PATH_INFO: `/v3/services` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:09.779401 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a356a-275e-4dba-b854-7330949f6cdd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:09.779947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a356a-275e-4dba-b854-7330949f6cdd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:09.781547 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a356a-275e-4dba-b854-7330949f6cdd None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:09.787659 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 352/717] 10.222.0.49 () {56 vars in 1124 bytes} [Mon Sep 26 10:20:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:09.793403 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a356a-275e-4dba-b854-7330949f6cdd None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:09.914732 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5f469-a3d1-4c6f-b463-b8a9ae064e29 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:09.951095 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f5f32-ce79-429e-b5b6-bb34fd5b6152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:09.951760 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f5f32-ce79-429e-b5b6-bb34fd5b6152 None admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:09.952040 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f5f32-ce79-429e-b5b6-bb34fd5b6152 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:09.952313 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f5f32-ce79-429e-b5b6-bb34fd5b6152 None admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:09.952993 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f5f32-ce79-429e-b5b6-bb34fd5b6152 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:09.954591 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f5f32-ce79-429e-b5b6-bb34fd5b6152 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:09.993567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 366/718] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 10:20:09 2022] POST /identity/v3/users => generated 254 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:20:10.013895 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a356a-275e-4dba-b854-7330949f6cdd None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:10.077057 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b0b56b-a9ba-4a52-82ab-b5ac9dafebe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:10.077941 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0b56b-a9ba-4a52-82ab-b5ac9dafebe5 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:10.078391 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0b56b-a9ba-4a52-82ab-b5ac9dafebe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:10.078809 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0b56b-a9ba-4a52-82ab-b5ac9dafebe5 None admin] PATH_INFO: `/v3/services` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:10.079670 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b0b56b-a9ba-4a52-82ab-b5ac9dafebe5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:10.080149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b0b56b-a9ba-4a52-82ab-b5ac9dafebe5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:10.081965 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b0b56b-a9ba-4a52-82ab-b5ac9dafebe5 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:10.087937 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 353/719] 10.222.0.49 () {56 vars in 1124 bytes} [Mon Sep 26 10:20:09 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:10.783900 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da8c26-64d3-4c78-9a4b-e2974519bad2 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:10.784177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da8c26-64d3-4c78-9a4b-e2974519bad2 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:10.784455 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da8c26-64d3-4c78-9a4b-e2974519bad2 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:10.786062 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 367/720] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:10.793175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7b62c-09e4-4c81-8938-6c9ab1c9418f None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:10.793175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7b62c-09e4-4c81-8938-6c9ab1c9418f None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:10.793175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7b62c-09e4-4c81-8938-6c9ab1c9418f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:10.819665 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-5ca7b62c-09e4-4c81-8938-6c9ab1c9418f None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:10.820930 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-5ca7b62c-09e4-4c81-8938-6c9ab1c9418f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:10.872515 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:10.931666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 354/721] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:20:11.355486 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da8c26-64d3-4c78-9a4b-e2974519bad2 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:11.359299 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:11.490766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da8c26-64d3-4c78-9a4b-e2974519bad2 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:11.507619 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce861a-c446-4b3e-9e18-7144f5600eb9 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:11.507912 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce861a-c446-4b3e-9e18-7144f5600eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:11.508190 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce861a-c446-4b3e-9e18-7144f5600eb9 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:11.509777 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 355/722] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:11.515041 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05198f4-cb17-4305-9fd8-a1aa98eb9b8d None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:11.515327 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05198f4-cb17-4305-9fd8-a1aa98eb9b8d None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:11.515603 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05198f4-cb17-4305-9fd8-a1aa98eb9b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:11.542925 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d6ead-c2e6-456c-a9db-3a3cdd09b1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:11.543576 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d6ead-c2e6-456c-a9db-3a3cdd09b1c6 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:11.543853 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d6ead-c2e6-456c-a9db-3a3cdd09b1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:11.544212 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d6ead-c2e6-456c-a9db-3a3cdd09b1c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:11.552966 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-e05198f4-cb17-4305-9fd8-a1aa98eb9b8d None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:11.554204 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-e05198f4-cb17-4305-9fd8-a1aa98eb9b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:11.566338 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d6ead-c2e6-456c-a9db-3a3cdd09b1c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:20:11.571141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-132d6ead-c2e6-456c-a9db-3a3cdd09b1c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:20:11.571141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:11.571141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:11.571141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:11.571141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:11.571141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:11.571141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:11.571141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:11.571141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:11.571141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:11.571141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:11.571141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:11.571141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:20:11.571141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:20:11.571141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:20:11.571141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:20:11.572046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:20:11.572046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:20:11.572046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:20:11.572046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:20:11.572046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:11.572046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:11.572046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:11.572046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:11.572046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:11.572046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:11.572046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:11.572046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:11.572046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:11.572046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:11.572046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:11.572046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:11.572918 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:11.572918 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:11.572918 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:11.572918 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:11.572918 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:20:11.572918 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:20:11.572918 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:20:11.572918 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:20:11.572918 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:20:11.572918 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:20:11.572918 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:20:11.572918 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:11.574128 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 368/723] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:20:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:11.579954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d6ead-c2e6-456c-a9db-3a3cdd09b1c6 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:11.701412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 356/724] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:20:11.709426 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:11.755925 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d6ead-c2e6-456c-a9db-3a3cdd09b1c6 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:11.802059 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb74d9-c4b5-4508-b5c7-b0923f575200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:11.802059 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb74d9-c4b5-4508-b5c7-b0923f575200 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:11.802059 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb74d9-c4b5-4508-b5c7-b0923f575200 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:11.802059 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb74d9-c4b5-4508-b5c7-b0923f575200 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:11.802464 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eb74d9-c4b5-4508-b5c7-b0923f575200 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:11.802464 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eb74d9-c4b5-4508-b5c7-b0923f575200 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:11.803321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eb74d9-c4b5-4508-b5c7-b0923f575200 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:11.822522 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 369/725] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:20:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:11.828341 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05198f4-cb17-4305-9fd8-a1aa98eb9b8d None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:11.846792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4c002-89f3-4b9a-9c4b-02c1ea2a04bb None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:11.847289 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4c002-89f3-4b9a-9c4b-02c1ea2a04bb None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:11.847780 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4c002-89f3-4b9a-9c4b-02c1ea2a04bb None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:11.850152 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 370/726] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:11.859821 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019f51a-2d56-40e7-a147-1ea37922c0a9 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:11.860546 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019f51a-2d56-40e7-a147-1ea37922c0a9 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:11.860970 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019f51a-2d56-40e7-a147-1ea37922c0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:11.899211 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-6019f51a-2d56-40e7-a147-1ea37922c0a9 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:11.900659 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-6019f51a-2d56-40e7-a147-1ea37922c0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:12.075956 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 371/727] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:20:12.085478 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6019f51a-2d56-40e7-a147-1ea37922c0a9 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:12.108913 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:12.126602 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05198f4-cb17-4305-9fd8-a1aa98eb9b8d None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:12.225293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f1dfa-3f5d-413a-a397-d06bcb61e5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:12.226468 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f1dfa-3f5d-413a-a397-d06bcb61e5b7 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:12.227929 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f1dfa-3f5d-413a-a397-d06bcb61e5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:12.228528 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f1dfa-3f5d-413a-a397-d06bcb61e5b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:12.244092 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:12.250853 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f1dfa-3f5d-413a-a397-d06bcb61e5b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:12.253874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f1dfa-3f5d-413a-a397-d06bcb61e5b7 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:12.287519 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-1f9f1dfa-3f5d-413a-a397-d06bcb61e5b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:12.287519 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:12.287519 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:12.287519 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:12.287519 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:12.287519 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:12.287519 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:12.287519 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:12.287519 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:12.287519 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:12.287519 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:12.287519 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:12.287519 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:12.287519 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:12.287519 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:12.287519 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:12.288734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:12.288734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:12.288734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:12.288734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:12.288734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:12.288734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:12.288734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:12.288734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:12.288734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:12.288734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:12.288734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:12.288734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:12.288734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:12.288734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:12.288734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:12.288734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:12.288734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:12.289666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:12.289666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:12.289666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:12.300038 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 357/728] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 472 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:12.301115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f1dfa-3f5d-413a-a397-d06bcb61e5b7 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:12.341189 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6019f51a-2d56-40e7-a147-1ea37922c0a9 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:12.414223 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11628d73-9f30-414c-9ae8-2a89c3159e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:12.415655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11628d73-9f30-414c-9ae8-2a89c3159e5d None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:12.416149 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11628d73-9f30-414c-9ae8-2a89c3159e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:12.416800 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11628d73-9f30-414c-9ae8-2a89c3159e5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:12.429233 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11628d73-9f30-414c-9ae8-2a89c3159e5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:20:12.434332 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-11628d73-9f30-414c-9ae8-2a89c3159e5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:20:12.434332 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:12.434332 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:12.434332 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:12.434332 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:12.434332 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:12.434332 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:12.434332 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:12.434332 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:12.434332 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:12.434332 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:12.434332 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:12.434332 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:20:12.434332 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:20:12.434332 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:20:12.434332 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:20:12.435843 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:20:12.435843 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:20:12.435843 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:20:12.435843 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:20:12.435843 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:12.435843 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:12.435843 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:12.435843 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:12.435843 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:12.435843 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:12.435843 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:12.435843 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:12.435843 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:12.435843 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:12.435843 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:12.435843 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:12.436673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:12.436673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:12.436673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:12.436673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:12.436673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:20:12.436673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:20:12.436673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:20:12.436673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:20:12.436673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:20:12.436673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:20:12.436673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:20:12.436673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:12.439267 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 372/729] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:20:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:20:12.444691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11628d73-9f30-414c-9ae8-2a89c3159e5d None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:12.470595 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f1dfa-3f5d-413a-a397-d06bcb61e5b7 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:12.520777 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5c0ca-dba9-474e-8057-fe4cfe88d0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:12.521406 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5c0ca-dba9-474e-8057-fe4cfe88d0c6 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:12.521677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5c0ca-dba9-474e-8057-fe4cfe88d0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:12.521961 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5c0ca-dba9-474e-8057-fe4cfe88d0c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:12.522632 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e5c0ca-dba9-474e-8057-fe4cfe88d0c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:12.522938 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e5c0ca-dba9-474e-8057-fe4cfe88d0c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:12.524587 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e5c0ca-dba9-474e-8057-fe4cfe88d0c6 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:12.541795 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 358/730] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:12.542703 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5c0ca-dba9-474e-8057-fe4cfe88d0c6 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:12.553724 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:12.564663 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11628d73-9f30-414c-9ae8-2a89c3159e5d None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:12.581949 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:12.609628 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7537bef-3bc1-4b71-9f32-0ec3195b0327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:12.610389 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7537bef-3bc1-4b71-9f32-0ec3195b0327 None admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:12.610762 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7537bef-3bc1-4b71-9f32-0ec3195b0327 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:12.611177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7537bef-3bc1-4b71-9f32-0ec3195b0327 None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:12.612736 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7537bef-3bc1-4b71-9f32-0ec3195b0327 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:12.614634 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7537bef-3bc1-4b71-9f32-0ec3195b0327 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:12.642643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 373/731] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Sep 26 10:20:12 2022] POST /identity/v3/services => generated 240 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:20:12.642643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7537bef-3bc1-4b71-9f32-0ec3195b0327 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:12.736409 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5c0ca-dba9-474e-8057-fe4cfe88d0c6 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:12.784149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd21419-598c-4aa4-9ced-4aedeac8a11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:12.784765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd21419-598c-4aa4-9ced-4aedeac8a11f None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:12.785042 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd21419-598c-4aa4-9ced-4aedeac8a11f None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:12.785313 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd21419-598c-4aa4-9ced-4aedeac8a11f None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:12.786105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd21419-598c-4aa4-9ced-4aedeac8a11f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:12.786408 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd21419-598c-4aa4-9ced-4aedeac8a11f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:12.787995 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd21419-598c-4aa4-9ced-4aedeac8a11f None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:12.797282 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7537bef-3bc1-4b71-9f32-0ec3195b0327 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:12.810112 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 359/732] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:20:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:12.811723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3109278-9819-4e21-bdf5-48ae97636150 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:12.812159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3109278-9819-4e21-bdf5-48ae97636150 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:12.812598 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3109278-9819-4e21-bdf5-48ae97636150 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:12.814499 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 360/733] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:12.816012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3109278-9819-4e21-bdf5-48ae97636150 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:12.844294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98e1bc1-ccff-44c6-a703-d83a4056f9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:12.844929 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e1bc1-ccff-44c6-a703-d83a4056f9fc None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:12.845206 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e1bc1-ccff-44c6-a703-d83a4056f9fc None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:12.845476 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e1bc1-ccff-44c6-a703-d83a4056f9fc None admin] PATH_INFO: `/v3/users/swift` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:12.854001 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e1bc1-ccff-44c6-a703-d83a4056f9fc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:12.855504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e1bc1-ccff-44c6-a703-d83a4056f9fc None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:12.866768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-b98e1bc1-ccff-44c6-a703-d83a4056f9fc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 26 10:20:12.866768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:12.866768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:12.866768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:12.866768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:12.866768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:12.866768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:12.866768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:12.866768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:12.866768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:12.866768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:12.866768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:12.866768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:20:12.866768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:20:12.866768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:20:12.866768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:20:12.867748 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:12.867748 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:12.867748 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:20:12.867748 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:12.867748 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:20:12.867748 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:12.867748 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:12.867748 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:12.867748 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:12.867748 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:12.867748 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:12.867748 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:12.867748 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:12.867748 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:12.867748 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:12.867748 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:12.867748 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:12.867748 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:12.868673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:12.868673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:12.868673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:20:12.868673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:20:12.868673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:20:12.868673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:20:12.868673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:20:12.868673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:20:12.868673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 26 10:20:12.868673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:12.868673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 374/734] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:20:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:12.869265 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1afcec-450b-4aa1-b89a-42b185f40ea2 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:12.869265 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1afcec-450b-4aa1-b89a-42b185f40ea2 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:12.869861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1afcec-450b-4aa1-b89a-42b185f40ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:12.888620 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-3f1afcec-450b-4aa1-b89a-42b185f40ea2 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:12.889853 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-3f1afcec-450b-4aa1-b89a-42b185f40ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:12.892778 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:12.919744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:12.937172 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3109278-9819-4e21-bdf5-48ae97636150 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:12.966674 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 375/735] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:12 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:20:12.968089 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5d9c0-1039-4bb7-84ff-03b1052bcec8 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:12.968438 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5d9c0-1039-4bb7-84ff-03b1052bcec8 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:12.968771 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5d9c0-1039-4bb7-84ff-03b1052bcec8 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:12.970321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 376/736] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:12.970885 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c5d9c0-1039-4bb7-84ff-03b1052bcec8 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:12.993318 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccac3623-26a8-484d-b4c8-294d974385fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:12.993947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccac3623-26a8-484d-b4c8-294d974385fa None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:12.994249 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccac3623-26a8-484d-b4c8-294d974385fa None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:12.994518 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccac3623-26a8-484d-b4c8-294d974385fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:13.008053 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccac3623-26a8-484d-b4c8-294d974385fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:13.009542 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccac3623-26a8-484d-b4c8-294d974385fa None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:13.024934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-ccac3623-26a8-484d-b4c8-294d974385fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:13.024934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:13.024934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:13.024934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:13.024934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:13.024934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:13.024934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:13.024934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:13.024934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:13.024934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:13.024934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:13.024934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:13.024934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:13.024934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:13.024934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:13.024934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:13.025919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:13.025919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:13.025919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:13.025919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:13.025919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:13.025919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:13.025919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:13.025919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:13.025919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:13.025919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:13.025919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:13.025919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:13.025919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:13.025919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:13.025919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:13.025919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:13.025919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:13.026924 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:13.026924 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:13.026924 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:13.026924 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 361/737] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:13.027605 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d662fcb-b9a8-48e4-b157-1654e6ed90ce None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:13.027866 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d662fcb-b9a8-48e4-b157-1654e6ed90ce None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:13.028174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d662fcb-b9a8-48e4-b157-1654e6ed90ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:13.051320 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-4d662fcb-b9a8-48e4-b157-1654e6ed90ce None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:13.053393 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-4d662fcb-b9a8-48e4-b157-1654e6ed90ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:13.104204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c5d9c0-1039-4bb7-84ff-03b1052bcec8 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:13.146147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 362/738] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:13 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 10:20:13.146147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d662fcb-b9a8-48e4-b157-1654e6ed90ce None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:13.158708 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0737704-1619-44d7-be97-76e5cd8178d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:13.159585 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0737704-1619-44d7-be97-76e5cd8178d3 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:13.160065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0737704-1619-44d7-be97-76e5cd8178d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:13.160480 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0737704-1619-44d7-be97-76e5cd8178d3 None admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:13.161422 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0737704-1619-44d7-be97-76e5cd8178d3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:13.161878 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0737704-1619-44d7-be97-76e5cd8178d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:13.164441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0737704-1619-44d7-be97-76e5cd8178d3 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:13.205068 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:13.205504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 377/739] 10.222.0.49 () {56 vars in 1135 bytes} [Mon Sep 26 10:20:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:13.212780 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0737704-1619-44d7-be97-76e5cd8178d3 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:13.238807 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:13.290956 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d662fcb-b9a8-48e4-b157-1654e6ed90ce None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:13.320474 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0737704-1619-44d7-be97-76e5cd8178d3 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:13.333206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb41d2-5497-4ff9-9b8d-7d76f011943a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:13.333815 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb41d2-5497-4ff9-9b8d-7d76f011943a None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:13.334111 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb41d2-5497-4ff9-9b8d-7d76f011943a None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:13.334650 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb41d2-5497-4ff9-9b8d-7d76f011943a None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:13.335296 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb41d2-5497-4ff9-9b8d-7d76f011943a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:13.335957 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb41d2-5497-4ff9-9b8d-7d76f011943a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:13.337567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb41d2-5497-4ff9-9b8d-7d76f011943a None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:13.352605 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 363/740] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:13.353388 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb41d2-5497-4ff9-9b8d-7d76f011943a None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:13.362184 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f15305-6abe-4ada-b0eb-a4b5356bc3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:13.362875 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f15305-6abe-4ada-b0eb-a4b5356bc3c1 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:13.363260 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f15305-6abe-4ada-b0eb-a4b5356bc3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:13.363595 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f15305-6abe-4ada-b0eb-a4b5356bc3c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:13.444656 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f15305-6abe-4ada-b0eb-a4b5356bc3c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:13.446416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f15305-6abe-4ada-b0eb-a4b5356bc3c1 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:13.471158 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb41d2-5497-4ff9-9b8d-7d76f011943a None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:13.472077 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:13.478497 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:13.508389 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2622868d-b5bf-49aa-8954-dda73dae9309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:13.508990 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622868d-b5bf-49aa-8954-dda73dae9309 None admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:13.509265 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622868d-b5bf-49aa-8954-dda73dae9309 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:13.509523 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622868d-b5bf-49aa-8954-dda73dae9309 None admin] PATH_INFO: `/v3/services` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:13.510194 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2622868d-b5bf-49aa-8954-dda73dae9309 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:13.511557 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2622868d-b5bf-49aa-8954-dda73dae9309 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:13.528112 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 364/741] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Sep 26 10:20:13 2022] POST /identity/v3/services => generated 251 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:20:13.529418 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2622868d-b5bf-49aa-8954-dda73dae9309 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:13.672129 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 378/742] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:20:13 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 10:20:13.672963 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2622868d-b5bf-49aa-8954-dda73dae9309 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:13.673093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f15305-6abe-4ada-b0eb-a4b5356bc3c1 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:13.712601 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b77ca-4173-44d3-b1d3-5d53a202d1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:13.713157 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b77ca-4173-44d3-b1d3-5d53a202d1f6 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:13.713431 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b77ca-4173-44d3-b1d3-5d53a202d1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:13.713698 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b77ca-4173-44d3-b1d3-5d53a202d1f6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:13.722591 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b77ca-4173-44d3-b1d3-5d53a202d1f6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:13.724057 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b77ca-4173-44d3-b1d3-5d53a202d1f6 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:13.737680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-f11b77ca-4173-44d3-b1d3-5d53a202d1f6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 10:20:13.737680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:13.737680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:13.737680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:13.737680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:13.737680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:13.737680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:13.737680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:13.737680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:13.737680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:13.737680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:13.737680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:13.737680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:20:13.737680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:20:13.737680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:20:13.737680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:20:13.738728 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:13.738728 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:13.738728 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:20:13.738728 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:13.738728 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:20:13.738728 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:13.738728 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:13.738728 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:13.738728 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:13.738728 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:13.738728 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:13.738728 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:13.738728 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:13.738728 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:13.738728 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:13.738728 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:13.738728 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:13.738728 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:13.739683 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:13.739683 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:13.739683 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:20:13.739683 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:20:13.739683 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:20:13.739683 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:20:13.739683 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:20:13.739683 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:20:13.739683 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 10:20:13.739683 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:13.743882 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 365/743] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Sep 26 10:20:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:13.745041 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b77ca-4173-44d3-b1d3-5d53a202d1f6 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:13.776343 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:13.817279 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f15305-6abe-4ada-b0eb-a4b5356bc3c1 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:13.851235 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:13.864993 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb0cca-9f14-48ab-9674-23de8e346e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:13.866331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb0cca-9f14-48ab-9674-23de8e346e16 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:13.867001 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb0cca-9f14-48ab-9674-23de8e346e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:13.867424 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb0cca-9f14-48ab-9674-23de8e346e16 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:13.871815 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b77ca-4173-44d3-b1d3-5d53a202d1f6 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:13.880243 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb0cca-9f14-48ab-9674-23de8e346e16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:20:13.887113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-4bdb0cca-9f14-48ab-9674-23de8e346e16 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:20:13.887113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:13.887113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:13.887113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:13.887113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:13.887113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:13.887113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:13.887113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:13.887113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:13.887113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:13.887113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:13.887113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:13.887113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:20:13.887113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:20:13.887113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:20:13.887113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:20:13.888260 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:20:13.888260 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:20:13.888260 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:20:13.888260 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:20:13.888260 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:13.888260 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:13.888260 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:13.888260 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:13.888260 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:13.888260 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:13.888260 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:13.888260 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:13.888260 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:13.888260 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:13.888260 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:13.888260 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:13.889310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:13.889310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:13.889310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:13.889310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:13.889310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:20:13.889310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:20:13.889310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:20:13.889310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:20:13.889310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:20:13.889310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:20:13.889310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:20:13.889310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:13.896353 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 379/744] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 10:20:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:13.896353 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb0cca-9f14-48ab-9674-23de8e346e16 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:13.921281 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a77e8-a229-4f52-8a06-f39989c6ff69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:13.922756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a77e8-a229-4f52-8a06-f39989c6ff69 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:13.923240 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a77e8-a229-4f52-8a06-f39989c6ff69 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:13.923710 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a77e8-a229-4f52-8a06-f39989c6ff69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:13.935669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a77e8-a229-4f52-8a06-f39989c6ff69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:13.938470 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a77e8-a229-4f52-8a06-f39989c6ff69 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:13.956694 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-358a77e8-a229-4f52-8a06-f39989c6ff69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:13.956694 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:13.956694 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:13.956694 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:13.956694 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:13.956694 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:13.956694 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:13.956694 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:13.956694 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:13.956694 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:13.956694 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:13.956694 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:13.956694 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:13.956694 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:13.956694 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:13.956694 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:13.957674 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:13.957674 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:13.957674 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:13.957674 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:13.957674 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:13.957674 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:13.957674 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:13.957674 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:13.957674 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:13.957674 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:13.957674 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:13.957674 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:13.957674 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:13.957674 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:13.957674 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:13.957674 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:13.957674 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:13.959013 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:13.959013 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:13.959013 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:13.959013 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 366/745] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:13.959239 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a77e8-a229-4f52-8a06-f39989c6ff69 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:14.001211 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb0cca-9f14-48ab-9674-23de8e346e16 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:14.036721 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8074ca0-c256-49e5-9002-28927bc5a5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:14.037323 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8074ca0-c256-49e5-9002-28927bc5a5bb None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:14.037600 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8074ca0-c256-49e5-9002-28927bc5a5bb None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:14.037873 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8074ca0-c256-49e5-9002-28927bc5a5bb None admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:14.038623 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8074ca0-c256-49e5-9002-28927bc5a5bb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:14.038924 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8074ca0-c256-49e5-9002-28927bc5a5bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:14.040341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8074ca0-c256-49e5-9002-28927bc5a5bb None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:14.053607 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:14.059141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 380/746] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Sep 26 10:20:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:14.060526 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8074ca0-c256-49e5-9002-28927bc5a5bb None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:14.095656 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a77e8-a229-4f52-8a06-f39989c6ff69 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:14.105579 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:14.143178 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712c1c1-c9bf-421f-86dd-423eb3dfe054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:14.143792 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712c1c1-c9bf-421f-86dd-423eb3dfe054 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:14.144065 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712c1c1-c9bf-421f-86dd-423eb3dfe054 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:14.144330 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712c1c1-c9bf-421f-86dd-423eb3dfe054 None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:14.144996 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5712c1c1-c9bf-421f-86dd-423eb3dfe054 None admin] RBAC: Adding query filter params (name=service) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:14.145769 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5712c1c1-c9bf-421f-86dd-423eb3dfe054 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:14.147648 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5712c1c1-c9bf-421f-86dd-423eb3dfe054 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:14.159592 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 367/747] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 10:20:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:14.161186 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712c1c1-c9bf-421f-86dd-423eb3dfe054 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:14.172928 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8074ca0-c256-49e5-9002-28927bc5a5bb None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:14.212200 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e00fbc-626e-4827-bbb8-e14cd39f53fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:14.212831 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e00fbc-626e-4827-bbb8-e14cd39f53fe None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:14.213107 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e00fbc-626e-4827-bbb8-e14cd39f53fe None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:14.213372 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e00fbc-626e-4827-bbb8-e14cd39f53fe None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:14.214007 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e00fbc-626e-4827-bbb8-e14cd39f53fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:14.214327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e00fbc-626e-4827-bbb8-e14cd39f53fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:14.215675 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e00fbc-626e-4827-bbb8-e14cd39f53fe None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:14.232339 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 381/748] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:14.233194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e00fbc-626e-4827-bbb8-e14cd39f53fe None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:14.285927 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712c1c1-c9bf-421f-86dd-423eb3dfe054 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:14.322144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe197513-d910-431c-a63b-37a1608b44ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:14.323163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe197513-d910-431c-a63b-37a1608b44ed None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:14.323641 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe197513-d910-431c-a63b-37a1608b44ed None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:14.324133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe197513-d910-431c-a63b-37a1608b44ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:14.328428 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:14.339363 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe197513-d910-431c-a63b-37a1608b44ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:14.341776 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe197513-d910-431c-a63b-37a1608b44ed None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:14.357147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-fe197513-d910-431c-a63b-37a1608b44ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:14.357147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:14.357147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:14.357147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:14.357147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:14.357147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:14.357147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:14.357147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:14.357147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:14.357147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:14.357147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:14.357147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:14.357147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:14.357147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:14.357147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:14.357147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:14.358163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:14.358163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:14.358163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:14.358163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:14.358163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:14.358163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:14.358163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:14.358163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:14.358163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:14.358163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:14.358163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:14.358163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:14.358163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:14.358163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:14.358163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:14.358163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:14.358163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:14.359118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:14.359118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:14.359118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:14.360243 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:14.361588 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 368/749] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:14.362830 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe197513-d910-431c-a63b-37a1608b44ed None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:14.363870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e00fbc-626e-4827-bbb8-e14cd39f53fe None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:14.411295 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f338814-7d3c-4b0b-8795-e7c8084fe7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:14.411932 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f338814-7d3c-4b0b-8795-e7c8084fe7f3 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:14.412230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f338814-7d3c-4b0b-8795-e7c8084fe7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:14.412752 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f338814-7d3c-4b0b-8795-e7c8084fe7f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:14.427946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f338814-7d3c-4b0b-8795-e7c8084fe7f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:14.429425 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f338814-7d3c-4b0b-8795-e7c8084fe7f3 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:14.448965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-8f338814-7d3c-4b0b-8795-e7c8084fe7f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:14.448965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:14.448965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:14.448965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:14.448965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:14.448965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:14.448965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:14.448965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:14.448965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:14.448965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:14.448965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:14.448965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:14.448965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:14.448965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:14.448965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:14.448965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:14.451177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:14.451177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:14.451177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:14.451177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:14.451177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:14.451177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:14.451177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:14.451177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:14.451177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:14.451177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:14.451177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:14.451177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:14.451177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:14.451177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:14.451177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:14.451177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:14.451177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:14.452093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:14.452093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:14.452093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:14.454865 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 382/750] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:14.455901 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f338814-7d3c-4b0b-8795-e7c8084fe7f3 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:14.496927 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe197513-d910-431c-a63b-37a1608b44ed None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:14.541131 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a994b-2f31-44e4-850d-4c065881217d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:14.541775 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a994b-2f31-44e4-850d-4c065881217d None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:14.542100 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a994b-2f31-44e4-850d-4c065881217d None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:14.542379 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a994b-2f31-44e4-850d-4c065881217d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:14.550552 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28a994b-2f31-44e4-850d-4c065881217d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:14.551930 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28a994b-2f31-44e4-850d-4c065881217d None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:14.564621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-f28a994b-2f31-44e4-850d-4c065881217d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:14.564621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:14.564621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:14.564621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:14.564621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:14.564621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:14.564621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:14.564621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:14.564621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:14.564621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:14.564621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:14.564621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:14.564621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:20:14.564621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:20:14.564621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:20:14.565570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:20:14.565570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:14.565570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:14.565570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:14.565570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:14.565570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:14.565570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:14.565570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:14.565570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:14.565570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:14.565570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:14.565570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:14.565570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:14.565570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:14.565570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:14.565570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:14.565570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:14.566513 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:20:14.566513 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:20:14.566513 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:20:14.566513 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:20:14.566513 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:20:14.566513 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:20:14.566513 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:14.566513 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:14.566513 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 369/751] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:20:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:14.567444 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a994b-2f31-44e4-850d-4c065881217d None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:14.583077 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f338814-7d3c-4b0b-8795-e7c8084fe7f3 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:14.623968 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:14.629474 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d8e17-a413-4faf-b09d-b445cdd6dcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:14.630364 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d8e17-a413-4faf-b09d-b445cdd6dcaf None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:14.630789 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d8e17-a413-4faf-b09d-b445cdd6dcaf None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:14.631222 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d8e17-a413-4faf-b09d-b445cdd6dcaf None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:14.632108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d8e17-a413-4faf-b09d-b445cdd6dcaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:14.632685 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d8e17-a413-4faf-b09d-b445cdd6dcaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:14.634269 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d8e17-a413-4faf-b09d-b445cdd6dcaf None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:14.648320 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 383/752] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:14.649611 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d8e17-a413-4faf-b09d-b445cdd6dcaf None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:14.668395 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:14.691817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a994b-2f31-44e4-850d-4c065881217d None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:14.743763 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebff5eb-ee06-4a50-810e-32128d004381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:14.744372 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebff5eb-ee06-4a50-810e-32128d004381 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:14.744658 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebff5eb-ee06-4a50-810e-32128d004381 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:14.744934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebff5eb-ee06-4a50-810e-32128d004381 None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:14.745581 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebff5eb-ee06-4a50-810e-32128d004381 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:14.745892 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebff5eb-ee06-4a50-810e-32128d004381 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:14.748480 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebff5eb-ee06-4a50-810e-32128d004381 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:14.769854 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 370/753] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:14.769854 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebff5eb-ee06-4a50-810e-32128d004381 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:14.771518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d8e17-a413-4faf-b09d-b445cdd6dcaf None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:14.818702 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6578b08-792b-4c03-b452-2cabb78e8461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:14.819638 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6578b08-792b-4c03-b452-2cabb78e8461 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:14.820140 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6578b08-792b-4c03-b452-2cabb78e8461 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:14.820606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6578b08-792b-4c03-b452-2cabb78e8461 None admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:14.821490 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6578b08-792b-4c03-b452-2cabb78e8461 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:14.821991 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6578b08-792b-4c03-b452-2cabb78e8461 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:14.823628 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6578b08-792b-4c03-b452-2cabb78e8461 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:14.837023 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 384/754] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:20:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:14.838250 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6578b08-792b-4c03-b452-2cabb78e8461 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:14.893794 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebff5eb-ee06-4a50-810e-32128d004381 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:14.904851 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:14.932767 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e0c375-e4a4-4f2d-ad0d-64711ec4f320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:14.934245 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0c375-e4a4-4f2d-ad0d-64711ec4f320 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:14.935266 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0c375-e4a4-4f2d-ad0d-64711ec4f320 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:14.935753 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0c375-e4a4-4f2d-ad0d-64711ec4f320 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:14.938330 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:14.944503 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e0c375-e4a4-4f2d-ad0d-64711ec4f320 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:14.946995 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e0c375-e4a4-4f2d-ad0d-64711ec4f320 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:14.962474 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-b9e0c375-e4a4-4f2d-ad0d-64711ec4f320 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:14.962474 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:14.962474 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:14.962474 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:14.962474 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:14.962474 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:14.962474 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:14.962474 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:14.962474 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:14.962474 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:14.962474 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:14.962474 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:14.962474 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:20:14.962474 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:20:14.962474 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:20:14.963384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:20:14.963384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:14.963384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:14.963384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:14.963384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:14.963384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:14.963384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:14.963384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:14.963384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:14.963384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:14.963384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:14.963384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:14.963384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:14.963384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:14.963384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:14.963384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:14.963384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:14.964268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:20:14.964268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:20:14.964268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:20:14.964268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:20:14.964268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:20:14.964268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:20:14.964268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:14.964268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:14.967076 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 371/755] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:20:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:14.968168 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e0c375-e4a4-4f2d-ad0d-64711ec4f320 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:14.973492 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6578b08-792b-4c03-b452-2cabb78e8461 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:15.015083 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d737e3-1b50-4941-8167-a46ae596b2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:15.015714 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d737e3-1b50-4941-8167-a46ae596b2b8 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:15.015994 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d737e3-1b50-4941-8167-a46ae596b2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:15.016269 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d737e3-1b50-4941-8167-a46ae596b2b8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:15.023872 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d737e3-1b50-4941-8167-a46ae596b2b8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:15.025311 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d737e3-1b50-4941-8167-a46ae596b2b8 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:15.036840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-c6d737e3-1b50-4941-8167-a46ae596b2b8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 10:20:15.036840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:15.036840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:15.036840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:15.036840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:15.036840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:15.036840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:15.036840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:15.036840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:15.036840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:15.036840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:15.036840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:15.036840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:20:15.036840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:20:15.036840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:20:15.036840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:20:15.037805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:15.037805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:15.037805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:20:15.037805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:15.037805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:20:15.037805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:15.037805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:15.037805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:15.037805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:15.037805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:15.037805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:15.037805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:15.037805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:15.037805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:15.037805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:15.037805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:15.037805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:15.037805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:15.039100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:15.039100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:15.039100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:20:15.039100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:20:15.039100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:20:15.039100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:20:15.039100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:20:15.039100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:20:15.039100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 10:20:15.039100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:15.039100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 385/756] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Sep 26 10:20:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:15.039100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d737e3-1b50-4941-8167-a46ae596b2b8 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:15.087650 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e0c375-e4a4-4f2d-ad0d-64711ec4f320 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:15.125599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a7c040-aa35-44a0-9407-8d0af11a0f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:15.126579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7c040-aa35-44a0-9407-8d0af11a0f4a None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:15.127056 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7c040-aa35-44a0-9407-8d0af11a0f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:15.127521 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7c040-aa35-44a0-9407-8d0af11a0f4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:15.128588 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a7c040-aa35-44a0-9407-8d0af11a0f4a None admin] RBAC: Adding query filter params (role.id=532b37d9076c45618cf352aafb85f35e, scope.project.id=cff3a6011f4c4893bddffdad3d69f3df, user.id=5e6be34fc93a4551a83e5b8380188a6b) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:15.129123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a7c040-aa35-44a0-9407-8d0af11a0f4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:15.130766 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a7c040-aa35-44a0-9407-8d0af11a0f4a None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:15.146049 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 372/757] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:20:14 2022] GET /identity/v3/role_assignments?user.id=5e6be34fc93a4551a83e5b8380188a6b&scope.project.id=cff3a6011f4c4893bddffdad3d69f3df&role.id=532b37d9076c45618cf352aafb85f35e => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:15.146049 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92f22f-c493-4389-83ae-1620a3bdbb9a None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:15.146049 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92f22f-c493-4389-83ae-1620a3bdbb9a None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:15.146399 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92f22f-c493-4389-83ae-1620a3bdbb9a None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:15.148572 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 373/758] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:15.150128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92f22f-c493-4389-83ae-1620a3bdbb9a None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:15.163320 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d737e3-1b50-4941-8167-a46ae596b2b8 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:15.198776 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:15.202334 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:15.209003 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461f464-7dd6-4dbf-8808-ffd8cab8ac02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:15.210067 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461f464-7dd6-4dbf-8808-ffd8cab8ac02 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:15.210549 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461f464-7dd6-4dbf-8808-ffd8cab8ac02 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:15.211018 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461f464-7dd6-4dbf-8808-ffd8cab8ac02 None admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:15.212048 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f461f464-7dd6-4dbf-8808-ffd8cab8ac02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:15.212623 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f461f464-7dd6-4dbf-8808-ffd8cab8ac02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:15.214552 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f461f464-7dd6-4dbf-8808-ffd8cab8ac02 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:15.229380 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 386/759] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:20:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:15.236134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9b730-59d6-4254-b99b-e2f24ad59f75 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:15.236515 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9b730-59d6-4254-b99b-e2f24ad59f75 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:15.236957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9b730-59d6-4254-b99b-e2f24ad59f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:15.257456 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-abb9b730-59d6-4254-b99b-e2f24ad59f75 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:15.259355 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-abb9b730-59d6-4254-b99b-e2f24ad59f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:15.285211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92f22f-c493-4389-83ae-1620a3bdbb9a None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:15.326819 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b52acd-839d-4fa9-9b23-c16060bf3766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:15.327479 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b52acd-839d-4fa9-9b23-c16060bf3766 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:15.327758 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b52acd-839d-4fa9-9b23-c16060bf3766 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:15.328031 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b52acd-839d-4fa9-9b23-c16060bf3766 None admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:15.328779 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b52acd-839d-4fa9-9b23-c16060bf3766 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:15.329085 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b52acd-839d-4fa9-9b23-c16060bf3766 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:15.330586 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b52acd-839d-4fa9-9b23-c16060bf3766 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:15.354149 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 387/760] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:20:15.354149 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9b730-59d6-4254-b99b-e2f24ad59f75 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:15.355508 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 374/761] 10.222.0.49 () {56 vars in 1149 bytes} [Mon Sep 26 10:20:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:15.361414 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b52acd-839d-4fa9-9b23-c16060bf3766 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:15.520255 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9b730-59d6-4254-b99b-e2f24ad59f75 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:15.544965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:15.554182 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:15.557736 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e55599-6561-43fe-8f86-9a7cd8c7c52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:15.558436 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e55599-6561-43fe-8f86-9a7cd8c7c52d None admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:15.558721 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e55599-6561-43fe-8f86-9a7cd8c7c52d None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:15.559270 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e55599-6561-43fe-8f86-9a7cd8c7c52d None admin] PATH_INFO: `/v3/projects/cff3a6011f4c4893bddffdad3d69f3df/users/cee41014d5d043888c6b399710bccab4/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:15.562002 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e55599-6561-43fe-8f86-9a7cd8c7c52d None admin] RBAC: Authorizing `identity:create_grant(project_id=cff3a6011f4c4893bddffdad3d69f3df, user_id=cee41014d5d043888c6b399710bccab4, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:15.563469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e55599-6561-43fe-8f86-9a7cd8c7c52d None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:15.583522 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 388/762] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 10:20:15 2022] PUT /identity/v3/projects/cff3a6011f4c4893bddffdad3d69f3df/users/cee41014d5d043888c6b399710bccab4/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:20:15.584859 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b52acd-839d-4fa9-9b23-c16060bf3766 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:15.647899 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80958b9f-a2a4-432c-b01a-5e73ca2ad609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:15.648437 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80958b9f-a2a4-432c-b01a-5e73ca2ad609 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:15.648710 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80958b9f-a2a4-432c-b01a-5e73ca2ad609 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:15.648978 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80958b9f-a2a4-432c-b01a-5e73ca2ad609 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:15.662813 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e55599-6561-43fe-8f86-9a7cd8c7c52d None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:15.775647 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80958b9f-a2a4-432c-b01a-5e73ca2ad609 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:15.777459 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80958b9f-a2a4-432c-b01a-5e73ca2ad609 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:15.843654 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e55599-6561-43fe-8f86-9a7cd8c7c52d None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:15.888908 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c13531-2899-4f5a-8e32-77b4e9902e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:15.889549 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c13531-2899-4f5a-8e32-77b4e9902e62 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:15.889822 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c13531-2899-4f5a-8e32-77b4e9902e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:15.890116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c13531-2899-4f5a-8e32-77b4e9902e62 None admin] PATH_INFO: `/v3/services/network` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:15.890703 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c13531-2899-4f5a-8e32-77b4e9902e62 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:15.893044 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c13531-2899-4f5a-8e32-77b4e9902e62 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:15.905953 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-94c13531-2899-4f5a-8e32-77b4e9902e62 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 10:20:15.905953 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:15.905953 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 10:20:15.905953 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 10:20:15.905953 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 10:20:15.905953 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 10:20:15.905953 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 10:20:15.905953 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:15.905953 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 10:20:15.905953 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:15.905953 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 10:20:15.905953 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:15.905953 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:15.905953 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:15.905953 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:15.905953 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:15.905953 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:15.907002 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:15.907002 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:15.907002 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:15.907002 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:15.907002 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:15.907002 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:15.907002 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 10:20:15.907002 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 10:20:15.907002 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 10:20:15.907002 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 10:20:15.907002 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:15.907002 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:15.907002 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:15.907002 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:15.907002 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:15.907002 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:15.907853 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:15.907853 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:15.907853 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:15.907853 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:15.907853 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:15.907853 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:15.907853 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:15.907853 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:15.907853 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:15.907853 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:15.907853 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 10:20:15.907853 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:15.907853 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 10:20:15.907853 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:15.907853 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 389/763] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:20:15 2022] GET /identity/v3/services/network => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:15.914202 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c13531-2899-4f5a-8e32-77b4e9902e62 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:15.922578 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:15.937778 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 375/764] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:20:15 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 26 10:20:15.939794 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95904449-4f3c-4406-976b-c7cda71bb7f5 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:15.940288 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95904449-4f3c-4406-976b-c7cda71bb7f5 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:15.940737 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95904449-4f3c-4406-976b-c7cda71bb7f5 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:15.942513 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 376/765] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:15.947074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95904449-4f3c-4406-976b-c7cda71bb7f5 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:16.022435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:16.032023 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c13531-2899-4f5a-8e32-77b4e9902e62 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:16.052225 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95904449-4f3c-4406-976b-c7cda71bb7f5 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:16.071534 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70b64d-eb39-443d-995c-300b5858b52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:16.072196 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70b64d-eb39-443d-995c-300b5858b52f None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:16.072471 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70b64d-eb39-443d-995c-300b5858b52f None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:16.072784 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70b64d-eb39-443d-995c-300b5858b52f None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:16.073414 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70b64d-eb39-443d-995c-300b5858b52f None admin] RBAC: Adding query filter params (name=network) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:16.073711 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70b64d-eb39-443d-995c-300b5858b52f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:16.075155 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70b64d-eb39-443d-995c-300b5858b52f None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:16.080689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 390/766] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Sep 26 10:20:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:16.086804 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27d954-fc79-4e97-83e6-988fcd5c70c5 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:16.087083 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27d954-fc79-4e97-83e6-988fcd5c70c5 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:16.087362 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27d954-fc79-4e97-83e6-988fcd5c70c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:16.092066 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a4361-61c4-44cd-8b21-34dfbdb5a99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:16.092909 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a4361-61c4-44cd-8b21-34dfbdb5a99e None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:16.093364 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a4361-61c4-44cd-8b21-34dfbdb5a99e None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:16.093828 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a4361-61c4-44cd-8b21-34dfbdb5a99e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:16.105788 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890a4361-61c4-44cd-8b21-34dfbdb5a99e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:16.108135 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890a4361-61c4-44cd-8b21-34dfbdb5a99e None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:16.109867 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-3d27d954-fc79-4e97-83e6-988fcd5c70c5 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:16.111247 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-3d27d954-fc79-4e97-83e6-988fcd5c70c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:16.125498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-890a4361-61c4-44cd-8b21-34dfbdb5a99e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:16.125498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:16.125498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:16.125498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:16.125498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:16.125498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:16.125498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:16.125498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:16.125498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:16.125498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:16.125498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:16.125498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:16.125498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:16.125498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:16.125498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:16.125498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:16.126620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:16.126620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:16.126620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:16.126620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:16.126620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:16.126620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:16.126620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:16.126620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:16.126620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:16.126620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:16.126620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:16.126620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:16.126620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:16.126620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:16.126620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:16.126620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:16.126620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:16.127514 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:16.127514 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:16.127514 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:16.129731 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 377/767] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:20:16.130723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a4361-61c4-44cd-8b21-34dfbdb5a99e None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:16.179976 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:16.198050 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 391/768] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:20:16.198050 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27d954-fc79-4e97-83e6-988fcd5c70c5 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:16.238383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:16.291814 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a4361-61c4-44cd-8b21-34dfbdb5a99e None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:16.298045 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27d954-fc79-4e97-83e6-988fcd5c70c5 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:16.332710 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46606a85-da69-427f-a7b6-18bbd0626c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:16.338056 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46606a85-da69-427f-a7b6-18bbd0626c73 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:16.338056 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46606a85-da69-427f-a7b6-18bbd0626c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:16.338056 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46606a85-da69-427f-a7b6-18bbd0626c73 None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:16.338056 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46606a85-da69-427f-a7b6-18bbd0626c73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:16.338056 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46606a85-da69-427f-a7b6-18bbd0626c73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:16.338056 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46606a85-da69-427f-a7b6-18bbd0626c73 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:16.348418 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 392/769] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:16.353906 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46606a85-da69-427f-a7b6-18bbd0626c73 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:16.362969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493124eb-fc01-4f00-acbf-44f14307d68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:16.363848 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493124eb-fc01-4f00-acbf-44f14307d68d None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:16.364300 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493124eb-fc01-4f00-acbf-44f14307d68d None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:16.364712 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493124eb-fc01-4f00-acbf-44f14307d68d None admin] PATH_INFO: `/v3/services` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:16.365486 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493124eb-fc01-4f00-acbf-44f14307d68d None admin] RBAC: Adding query filter params (type=network) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:16.365929 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493124eb-fc01-4f00-acbf-44f14307d68d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:16.367461 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493124eb-fc01-4f00-acbf-44f14307d68d None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:16.373469 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 378/770] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Sep 26 10:20:16 2022] GET /identity/v3/services?type=network => generated 350 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:16.390495 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493124eb-fc01-4f00-acbf-44f14307d68d None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:16.464985 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:16.467846 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46606a85-da69-427f-a7b6-18bbd0626c73 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:16.503794 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782eab5-dc8a-4cbd-89fa-9c5cbca1d7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:16.504426 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782eab5-dc8a-4cbd-89fa-9c5cbca1d7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:16.504755 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782eab5-dc8a-4cbd-89fa-9c5cbca1d7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:16.505385 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782eab5-dc8a-4cbd-89fa-9c5cbca1d7e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:16.512314 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3782eab5-dc8a-4cbd-89fa-9c5cbca1d7e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:16.514401 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3782eab5-dc8a-4cbd-89fa-9c5cbca1d7e0 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:16.526341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-3782eab5-dc8a-4cbd-89fa-9c5cbca1d7e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:16.526341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:16.526341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:16.526341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:16.526341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:16.526341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:16.526341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:16.526341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:16.526341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:16.526341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:16.526341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:16.526341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:16.526341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:20:16.526341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:20:16.526341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:20:16.527688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:20:16.527688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:16.527688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:16.527688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:16.527688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:16.527688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:16.527688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:16.527688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:16.527688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:16.527688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:16.527688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:16.527688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:16.527688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:16.527688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:16.527688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:16.527688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:16.527688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:16.528645 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:20:16.528645 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:20:16.528645 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:20:16.528645 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:20:16.528645 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:20:16.528645 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:20:16.528645 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:16.528645 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:16.528645 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 393/771] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:20:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:16.533320 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782eab5-dc8a-4cbd-89fa-9c5cbca1d7e0 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:16.614052 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493124eb-fc01-4f00-acbf-44f14307d68d None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:16.621745 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:16.643028 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782eab5-dc8a-4cbd-89fa-9c5cbca1d7e0 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:16.650843 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e143897-7843-4036-a25e-c652f840eec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:16.651460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e143897-7843-4036-a25e-c652f840eec4 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:16.651736 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e143897-7843-4036-a25e-c652f840eec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:16.652005 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e143897-7843-4036-a25e-c652f840eec4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:16.652805 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e143897-7843-4036-a25e-c652f840eec4 None admin] RBAC: Adding query filter params (interface=public, service_id=46eefc063d7e4fe2935a5d43d91c294c, region_id=RegionOne) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:16.653116 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e143897-7843-4036-a25e-c652f840eec4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:16.654701 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e143897-7843-4036-a25e-c652f840eec4 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:16.663116 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 379/772] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Sep 26 10:20:16 2022] GET /identity/v3/endpoints?service_id=46eefc063d7e4fe2935a5d43d91c294c&interface=public®ion_id=RegionOne => generated 194 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:16.664086 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e143897-7843-4036-a25e-c652f840eec4 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:16.696961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad04272-a5fa-4520-8d7a-bbaf90d182c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:16.697678 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad04272-a5fa-4520-8d7a-bbaf90d182c3 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:16.697953 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad04272-a5fa-4520-8d7a-bbaf90d182c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:16.698240 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad04272-a5fa-4520-8d7a-bbaf90d182c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:16.698931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad04272-a5fa-4520-8d7a-bbaf90d182c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:16.699230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad04272-a5fa-4520-8d7a-bbaf90d182c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:16.701180 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad04272-a5fa-4520-8d7a-bbaf90d182c3 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:16.718304 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 394/773] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:20:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:16.719061 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad04272-a5fa-4520-8d7a-bbaf90d182c3 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:16.743934 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:16.815972 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e143897-7843-4036-a25e-c652f840eec4 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:16.827084 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad04272-a5fa-4520-8d7a-bbaf90d182c3 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:16.851264 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dd85f2-c3ca-42a5-9f8f-dfe979c0cd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:16.852182 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd85f2-c3ca-42a5-9f8f-dfe979c0cd5e None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:16.852660 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd85f2-c3ca-42a5-9f8f-dfe979c0cd5e None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:16.853127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd85f2-c3ca-42a5-9f8f-dfe979c0cd5e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:16.853919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dd85f2-c3ca-42a5-9f8f-dfe979c0cd5e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:16.855508 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dd85f2-c3ca-42a5-9f8f-dfe979c0cd5e None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:16.863003 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2955e-0f38-4749-93a0-37f7bff508ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:16.863604 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2955e-0f38-4749-93a0-37f7bff508ad None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:16.863881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2955e-0f38-4749-93a0-37f7bff508ad None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:16.864151 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2955e-0f38-4749-93a0-37f7bff508ad None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:16.864744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f2955e-0f38-4749-93a0-37f7bff508ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:16.866181 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f2955e-0f38-4749-93a0-37f7bff508ad None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:16.867440 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-92dd85f2-c3ca-42a5-9f8f-dfe979c0cd5e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 10:20:16.867440 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:16.867440 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 10:20:16.867440 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 10:20:16.867440 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 10:20:16.867440 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 10:20:16.867440 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 10:20:16.867440 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:16.867440 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 10:20:16.867440 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:16.867440 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 10:20:16.867440 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:16.867440 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:16.867440 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:16.867440 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:16.867440 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:16.867440 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:16.868421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:16.868421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:16.868421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:16.868421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:16.868421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:16.868421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:16.868421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 10:20:16.868421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 10:20:16.868421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 10:20:16.868421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 10:20:16.868421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:16.868421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:16.868421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:16.868421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:16.868421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:16.868421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:16.869267 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:16.869267 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:16.869267 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:16.869267 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:16.869267 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:16.869267 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:16.869267 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:16.869267 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:16.869267 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:16.869267 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:16.869267 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 10:20:16.869267 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:16.869267 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 10:20:16.869267 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:16.877065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 395/775] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Sep 26 10:20:16 2022] GET /identity/v3/services => generated 772 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:16.877421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 380/775] 10.222.0.49 () {56 vars in 1129 bytes} [Mon Sep 26 10:20:16 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:16.878167 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2955e-0f38-4749-93a0-37f7bff508ad None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:16.879295 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dd85f2-c3ca-42a5-9f8f-dfe979c0cd5e None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:16.917560 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:17.036362 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dd85f2-c3ca-42a5-9f8f-dfe979c0cd5e None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:17.045038 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:17.057124 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2955e-0f38-4749-93a0-37f7bff508ad None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:17.102701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d18fd-8e15-4270-a5db-31117d336254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:17.103340 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d18fd-8e15-4270-a5db-31117d336254 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:17.103617 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d18fd-8e15-4270-a5db-31117d336254 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:17.103892 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d18fd-8e15-4270-a5db-31117d336254 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:17.104630 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420d18fd-8e15-4270-a5db-31117d336254 None admin] RBAC: Adding query filter params (role.id=d36e58344b604bcda1cee49065e8497e, scope.project.id=cff3a6011f4c4893bddffdad3d69f3df, user.id=9158557f1a2947e2815dc8d586d8b0c3) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:17.104935 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420d18fd-8e15-4270-a5db-31117d336254 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:17.106457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420d18fd-8e15-4270-a5db-31117d336254 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:17.113969 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 396/776] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:20:16 2022] GET /identity/v3/role_assignments?user.id=9158557f1a2947e2815dc8d586d8b0c3&scope.project.id=cff3a6011f4c4893bddffdad3d69f3df&role.id=d36e58344b604bcda1cee49065e8497e => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:17.141929 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb4053-5c53-4288-b3e8-396aecb781a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:17.142597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb4053-5c53-4288-b3e8-396aecb781a6 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:17.142876 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb4053-5c53-4288-b3e8-396aecb781a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:17.143146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb4053-5c53-4288-b3e8-396aecb781a6 None admin] PATH_INFO: `/v3/services` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:17.143798 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb4053-5c53-4288-b3e8-396aecb781a6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:17.144100 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb4053-5c53-4288-b3e8-396aecb781a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:17.145588 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb4053-5c53-4288-b3e8-396aecb781a6 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:17.151227 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 381/777] 10.222.0.49 () {56 vars in 1124 bytes} [Mon Sep 26 10:20:16 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:17.157689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d18fd-8e15-4270-a5db-31117d336254 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:17.275668 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d18fd-8e15-4270-a5db-31117d336254 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:17.323990 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b64f3-d643-44be-a331-e013dd0183c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:17.324948 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b64f3-d643-44be-a331-e013dd0183c2 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:17.325475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b64f3-d643-44be-a331-e013dd0183c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:17.325982 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b64f3-d643-44be-a331-e013dd0183c2 None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:17.326851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5b64f3-d643-44be-a331-e013dd0183c2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:17.327351 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5b64f3-d643-44be-a331-e013dd0183c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:17.329015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5b64f3-d643-44be-a331-e013dd0183c2 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:17.336728 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 397/778] 10.222.0.49 () {56 vars in 1124 bytes} [Mon Sep 26 10:20:17 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:17.342367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb4053-5c53-4288-b3e8-396aecb781a6 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:17.450918 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:17.455102 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb4053-5c53-4288-b3e8-396aecb781a6 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:17.506531 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf09cfa-81f4-412d-a42e-b59cdd209c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:17.507211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf09cfa-81f4-412d-a42e-b59cdd209c88 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:17.507493 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf09cfa-81f4-412d-a42e-b59cdd209c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:17.507762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf09cfa-81f4-412d-a42e-b59cdd209c88 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:17.508472 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf09cfa-81f4-412d-a42e-b59cdd209c88 None admin] RBAC: Adding query filter params (interface=public, service_id=73e489b206344c8aac8a6665d2e10358, region_id=RegionOne) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:17.508844 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf09cfa-81f4-412d-a42e-b59cdd209c88 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:17.510306 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf09cfa-81f4-412d-a42e-b59cdd209c88 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:17.518219 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 382/779] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Sep 26 10:20:17 2022] GET /identity/v3/endpoints?service_id=73e489b206344c8aac8a6665d2e10358&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:17.523852 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b64f3-d643-44be-a331-e013dd0183c2 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:17.543220 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:17.609601 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e115e-c58a-4b87-8b3d-d732d5934107 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:17.610336 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e115e-c58a-4b87-8b3d-d732d5934107 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:17.610954 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e115e-c58a-4b87-8b3d-d732d5934107 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:17.615093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 383/780] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:20:17.621191 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe368ee3-2380-4a60-a275-da33c47b46f4 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:17.621191 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe368ee3-2380-4a60-a275-da33c47b46f4 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:17.621191 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe368ee3-2380-4a60-a275-da33c47b46f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:17.659080 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-fe368ee3-2380-4a60-a275-da33c47b46f4 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:17.660306 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-fe368ee3-2380-4a60-a275-da33c47b46f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:17.670984 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b64f3-d643-44be-a331-e013dd0183c2 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:17.710656 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994dfb5a-180e-4ea2-b1c0-7b6635f52c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:17.711723 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994dfb5a-180e-4ea2-b1c0-7b6635f52c1d None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:17.712205 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994dfb5a-180e-4ea2-b1c0-7b6635f52c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:17.712709 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994dfb5a-180e-4ea2-b1c0-7b6635f52c1d None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:17.713765 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994dfb5a-180e-4ea2-b1c0-7b6635f52c1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:17.716209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994dfb5a-180e-4ea2-b1c0-7b6635f52c1d None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:17.722773 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 398/781] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Sep 26 10:20:17 2022] GET /identity/v3/services => generated 772 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:17.774057 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 384/782] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:20:17.981335 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229cf1af-f5d8-4181-9f7a-803beaf984b4 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:17.982063 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229cf1af-f5d8-4181-9f7a-803beaf984b4 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:17.982550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229cf1af-f5d8-4181-9f7a-803beaf984b4 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:17.984405 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 399/783] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:18.000328 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4f0d2-e99d-40a0-b66c-c47f81ff7019 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:18.000654 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4f0d2-e99d-40a0-b66c-c47f81ff7019 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:18.000875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4f0d2-e99d-40a0-b66c-c47f81ff7019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:18.021246 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-79f4f0d2-e99d-40a0-b66c-c47f81ff7019 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:18.022469 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-79f4f0d2-e99d-40a0-b66c-c47f81ff7019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:18.078534 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229cf1af-f5d8-4181-9f7a-803beaf984b4 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:18.110367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 385/784] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 10:20:18.173832 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:18.200524 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229cf1af-f5d8-4181-9f7a-803beaf984b4 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:18.236363 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ff855-1136-435b-a255-7ddba4b9d47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:18.236923 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ff855-1136-435b-a255-7ddba4b9d47c None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:18.237208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ff855-1136-435b-a255-7ddba4b9d47c None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:18.237483 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ff855-1136-435b-a255-7ddba4b9d47c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:18.248594 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272ff855-1136-435b-a255-7ddba4b9d47c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:20:18.253513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-272ff855-1136-435b-a255-7ddba4b9d47c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:20:18.253513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:18.253513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:18.253513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:18.253513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:18.253513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:18.253513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:18.253513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:18.253513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:18.253513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:18.253513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:18.253513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:18.253513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:20:18.253513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:20:18.253513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:20:18.253513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:20:18.254464 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:20:18.254464 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:20:18.254464 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:20:18.254464 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:20:18.254464 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:18.254464 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:18.254464 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:18.254464 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:18.254464 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:18.254464 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:18.254464 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:18.254464 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:18.254464 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:18.254464 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:18.254464 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:18.254464 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:18.255325 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:18.255325 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:18.255325 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:18.255325 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:18.255325 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:20:18.255325 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:20:18.255325 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:20:18.255325 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:20:18.255325 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:20:18.255325 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:20:18.255325 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:20:18.255325 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:18.259516 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 400/785] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:20:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:18.261088 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f4f0d2-e99d-40a0-b66c-c47f81ff7019 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:18.297633 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:18.404193 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f4f0d2-e99d-40a0-b66c-c47f81ff7019 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:18.423219 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ff855-1136-435b-a255-7ddba4b9d47c None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:18.468227 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5310c434-d80c-46cf-9d23-8138d69928ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:18.469202 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310c434-d80c-46cf-9d23-8138d69928ef None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:18.469675 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310c434-d80c-46cf-9d23-8138d69928ef None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:18.470179 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310c434-d80c-46cf-9d23-8138d69928ef None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:18.471043 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5310c434-d80c-46cf-9d23-8138d69928ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:18.471552 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5310c434-d80c-46cf-9d23-8138d69928ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:18.473273 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5310c434-d80c-46cf-9d23-8138d69928ef None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:18.494085 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 386/786] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:20:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:18.500676 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5310c434-d80c-46cf-9d23-8138d69928ef None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:18.599597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:18.601788 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ff855-1136-435b-a255-7ddba4b9d47c None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:18.634875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5310c434-d80c-46cf-9d23-8138d69928ef None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:18.653033 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ffac47-9ec7-42fa-8302-5cd1bba6cf27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:18.654234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ffac47-9ec7-42fa-8302-5cd1bba6cf27 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:18.654506 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ffac47-9ec7-42fa-8302-5cd1bba6cf27 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:18.654772 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ffac47-9ec7-42fa-8302-5cd1bba6cf27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:18.662627 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:18.678014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ffac47-9ec7-42fa-8302-5cd1bba6cf27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:20:18.683020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-d5ffac47-9ec7-42fa-8302-5cd1bba6cf27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:20:18.683020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:18.683020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:18.683020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:18.683020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:18.683020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:18.683020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:18.683020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:18.683020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:18.683020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:18.683020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:18.683020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:18.683020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:20:18.683020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:20:18.683020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:20:18.683020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:20:18.683931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:20:18.683931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:20:18.683931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:20:18.683931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:20:18.683931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:18.683931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:18.683931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:18.683931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:18.683931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:18.683931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:18.683931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:18.683931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:18.683931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:18.683931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:18.683931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:18.683931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:18.684820 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:18.684820 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:18.684820 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:18.684820 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:18.684820 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:20:18.684820 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:20:18.684820 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:20:18.684820 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:20:18.684820 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:20:18.684820 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:20:18.684820 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:20:18.684820 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:18.686104 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 401/787] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:20:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:18.694313 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ffac47-9ec7-42fa-8302-5cd1bba6cf27 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:18.703100 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c8308-8a2b-470b-adcb-0d4e006b9585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:18.703983 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c8308-8a2b-470b-adcb-0d4e006b9585 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:18.704457 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c8308-8a2b-470b-adcb-0d4e006b9585 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:18.704936 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c8308-8a2b-470b-adcb-0d4e006b9585 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:18.717793 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10c8308-8a2b-470b-adcb-0d4e006b9585 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:18.719571 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10c8308-8a2b-470b-adcb-0d4e006b9585 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:18.737176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-b10c8308-8a2b-470b-adcb-0d4e006b9585 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:18.737176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:18.737176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:18.737176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:18.737176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:18.737176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:18.737176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:18.737176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:18.737176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:18.737176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:18.737176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:18.737176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:18.737176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:18.737176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:18.737176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:18.737176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:18.738207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:18.738207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:18.738207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:18.738207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:18.738207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:18.738207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:18.738207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:18.738207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:18.738207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:18.738207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:18.738207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:18.738207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:18.738207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:18.738207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:18.738207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:18.738207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:18.738207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:18.739277 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:18.739277 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:18.739277 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:18.741394 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 387/788] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:20:18.746690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c8308-8a2b-470b-adcb-0d4e006b9585 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:18.895164 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c8308-8a2b-470b-adcb-0d4e006b9585 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:18.929781 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ffac47-9ec7-42fa-8302-5cd1bba6cf27 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:18.934167 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b915c-f5ad-41cf-bb25-6ce3959e5010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:18.935364 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b915c-f5ad-41cf-bb25-6ce3959e5010 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:18.936510 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b915c-f5ad-41cf-bb25-6ce3959e5010 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:18.936782 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b915c-f5ad-41cf-bb25-6ce3959e5010 None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:18.937420 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b915c-f5ad-41cf-bb25-6ce3959e5010 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:18.937720 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b915c-f5ad-41cf-bb25-6ce3959e5010 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:18.939101 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b915c-f5ad-41cf-bb25-6ce3959e5010 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:18.942093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:18.954699 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 388/789] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:18.960153 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b915c-f5ad-41cf-bb25-6ce3959e5010 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:19.001334 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0745095-73b9-49fd-9194-096dcad8fc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:19.001861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0745095-73b9-49fd-9194-096dcad8fc7c None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:19.002158 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0745095-73b9-49fd-9194-096dcad8fc7c None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:19.002427 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0745095-73b9-49fd-9194-096dcad8fc7c None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:19.003082 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0745095-73b9-49fd-9194-096dcad8fc7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:19.003379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0745095-73b9-49fd-9194-096dcad8fc7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:19.004731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0745095-73b9-49fd-9194-096dcad8fc7c None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:19.022359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 402/790] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:20:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:19.031768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0745095-73b9-49fd-9194-096dcad8fc7c None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:19.101796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b915c-f5ad-41cf-bb25-6ce3959e5010 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:19.145153 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778d605-db72-4b5f-a77e-009f1b8f0f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:19.146771 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778d605-db72-4b5f-a77e-009f1b8f0f58 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:19.147267 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778d605-db72-4b5f-a77e-009f1b8f0f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:19.147747 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778d605-db72-4b5f-a77e-009f1b8f0f58 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:19.156823 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c778d605-db72-4b5f-a77e-009f1b8f0f58 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:19.158623 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c778d605-db72-4b5f-a77e-009f1b8f0f58 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:19.166672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:19.174294 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-c778d605-db72-4b5f-a77e-009f1b8f0f58 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 26 10:20:19.174294 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:19.174294 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:19.174294 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:19.174294 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:19.174294 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:19.174294 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:19.174294 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:19.174294 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:19.174294 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:19.174294 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:19.174294 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:19.174294 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:20:19.174294 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:20:19.174294 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:20:19.174294 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:20:19.175213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:19.175213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:19.175213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:20:19.175213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:19.175213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:20:19.175213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:19.175213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:19.175213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:19.175213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:19.175213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:19.175213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:19.175213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:19.175213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:19.175213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:19.175213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:19.175213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:19.175213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:19.175213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:19.176169 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:19.176169 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:19.176169 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:20:19.176169 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:20:19.176169 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:20:19.176169 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:20:19.176169 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:20:19.176169 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:20:19.176169 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 26 10:20:19.176169 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:19.180291 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 389/791] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:20:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:20:19.182504 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778d605-db72-4b5f-a77e-009f1b8f0f58 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:19.266731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0745095-73b9-49fd-9194-096dcad8fc7c None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:19.319035 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369b7d04-30bf-4317-83c9-6704ca9d73ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:19.319711 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b7d04-30bf-4317-83c9-6704ca9d73ba None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:19.319994 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b7d04-30bf-4317-83c9-6704ca9d73ba None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:19.320270 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b7d04-30bf-4317-83c9-6704ca9d73ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:19.342294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369b7d04-30bf-4317-83c9-6704ca9d73ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:19.343777 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369b7d04-30bf-4317-83c9-6704ca9d73ba None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:19.377194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-369b7d04-30bf-4317-83c9-6704ca9d73ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:19.377194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:19.377194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:19.377194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:19.377194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:19.377194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:19.377194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:19.377194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:19.377194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:19.377194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:19.377194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:19.377194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:19.377194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:19.377194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:19.377194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:19.377194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:19.378232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:19.378232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:19.378232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:19.378232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:19.378232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:19.378232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:19.378232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:19.378232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:19.378232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:19.378232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:19.378232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:19.378232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:19.378232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:19.378232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:19.378232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:19.378232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:19.378232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:19.379109 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:19.379109 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:19.379109 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:19.381224 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 403/792] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:19.387390 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369b7d04-30bf-4317-83c9-6704ca9d73ba None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:19.486385 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:19.496602 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369b7d04-30bf-4317-83c9-6704ca9d73ba None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:19.508276 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:19.534845 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2769f09-5be3-46ca-a427-6c251e686e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:19.535794 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2769f09-5be3-46ca-a427-6c251e686e97 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:19.536267 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2769f09-5be3-46ca-a427-6c251e686e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:19.536751 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2769f09-5be3-46ca-a427-6c251e686e97 None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:19.537640 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2769f09-5be3-46ca-a427-6c251e686e97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:19.538258 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2769f09-5be3-46ca-a427-6c251e686e97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:19.539829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2769f09-5be3-46ca-a427-6c251e686e97 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:19.553284 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 404/793] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:19.559034 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2769f09-5be3-46ca-a427-6c251e686e97 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:19.582143 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778d605-db72-4b5f-a77e-009f1b8f0f58 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:19.664963 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2769f09-5be3-46ca-a427-6c251e686e97 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:19.667883 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fdc780-973b-4d96-84ef-1f7e913c266a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:19.668503 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fdc780-973b-4d96-84ef-1f7e913c266a None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:19.668779 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fdc780-973b-4d96-84ef-1f7e913c266a None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:19.669575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fdc780-973b-4d96-84ef-1f7e913c266a None admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:19.670841 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fdc780-973b-4d96-84ef-1f7e913c266a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:19.670841 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fdc780-973b-4d96-84ef-1f7e913c266a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:19.672065 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fdc780-973b-4d96-84ef-1f7e913c266a None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:19.714368 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 390/794] 10.222.0.49 () {56 vars in 1135 bytes} [Mon Sep 26 10:20:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 527 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:19.714368 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fdc780-973b-4d96-84ef-1f7e913c266a None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:19.718437 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104481de-a4af-4cd6-bfe2-546399aeb761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:19.719387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104481de-a4af-4cd6-bfe2-546399aeb761 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:19.719894 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104481de-a4af-4cd6-bfe2-546399aeb761 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:19.720376 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104481de-a4af-4cd6-bfe2-546399aeb761 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:19.737049 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104481de-a4af-4cd6-bfe2-546399aeb761 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:19.738526 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104481de-a4af-4cd6-bfe2-546399aeb761 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:19.761377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-104481de-a4af-4cd6-bfe2-546399aeb761 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 10:20:19.761377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:19.761377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:19.761377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:19.761377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:19.761377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:19.761377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:19.761377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:19.761377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:19.761377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:19.761377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:19.761377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:19.761377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:20:19.761377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:20:19.761377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:20:19.761377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:20:19.762384 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:19.762384 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:19.762384 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:20:19.762384 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:19.762384 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:20:19.762384 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:19.762384 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:19.762384 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:19.762384 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:19.762384 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:19.762384 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:19.762384 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:19.762384 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:19.762384 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:19.762384 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:19.762384 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:19.762384 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:19.762384 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:19.763386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:19.763386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:19.763386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:20:19.763386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:20:19.763386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:20:19.763386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:20:19.763386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:20:19.763386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:20:19.763386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 10:20:19.763386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:19.763386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 405/795] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Sep 26 10:20:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:19.771002 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104481de-a4af-4cd6-bfe2-546399aeb761 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:19.807725 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:19.869524 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fdc780-973b-4d96-84ef-1f7e913c266a None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:19.883942 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104481de-a4af-4cd6-bfe2-546399aeb761 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:19.884981 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:19.914166 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d8281b-500f-492f-816e-c282b14ef254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:19.914893 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8281b-500f-492f-816e-c282b14ef254 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:19.915290 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8281b-500f-492f-816e-c282b14ef254 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:19.915561 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8281b-500f-492f-816e-c282b14ef254 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:19.924350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988dbe2d-b2dd-441b-8fa0-e6ad696c3c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:19.924956 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dbe2d-b2dd-441b-8fa0-e6ad696c3c34 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:19.925634 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dbe2d-b2dd-441b-8fa0-e6ad696c3c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:19.925914 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dbe2d-b2dd-441b-8fa0-e6ad696c3c34 None admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:19.926968 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988dbe2d-b2dd-441b-8fa0-e6ad696c3c34 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:19.927248 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d8281b-500f-492f-816e-c282b14ef254 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:19.927354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988dbe2d-b2dd-441b-8fa0-e6ad696c3c34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:19.928762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d8281b-500f-492f-816e-c282b14ef254 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:19.929092 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988dbe2d-b2dd-441b-8fa0-e6ad696c3c34 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:19.943791 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-89d8281b-500f-492f-816e-c282b14ef254 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:19.943791 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:19.943791 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:19.943791 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:19.943791 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:19.943791 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:19.943791 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:19.943791 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:19.943791 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:19.943791 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:19.943791 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:19.943791 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:19.943791 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:19.943791 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:19.943791 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:19.943791 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:19.944759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:19.944759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:19.944759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:19.944759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:19.944759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:19.944759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:19.944759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:19.944759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:19.944759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:19.944759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:19.944759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:19.944759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:19.944759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:19.944759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:19.944759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:19.944759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:19.944759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:19.945663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:19.945663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:19.945663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:19.945663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 391/796] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:19.946442 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6c27b-d168-4ddd-bc0d-b8caa389d7d9 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:19.946442 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6c27b-d168-4ddd-bc0d-b8caa389d7d9 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:19.946599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6c27b-d168-4ddd-bc0d-b8caa389d7d9 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:19.948092 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 392/797] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:19.950175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d900d-5d16-424e-9be2-5823b54ebb14 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:19.950175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d900d-5d16-424e-9be2-5823b54ebb14 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:19.950175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d900d-5d16-424e-9be2-5823b54ebb14 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:19.951299 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 393/798] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:19.952030 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d900d-5d16-424e-9be2-5823b54ebb14 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:19.953417 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 406/799] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Sep 26 10:20:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:19.954902 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a22d4-8fff-4d0a-bfcc-c57e5e6a30a6 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:19.955411 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a22d4-8fff-4d0a-bfcc-c57e5e6a30a6 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:19.955942 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a22d4-8fff-4d0a-bfcc-c57e5e6a30a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:19.978895 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-c36a22d4-8fff-4d0a-bfcc-c57e5e6a30a6 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:19.980089 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-c36a22d4-8fff-4d0a-bfcc-c57e5e6a30a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:20.062089 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 407/800] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:20:20.063462 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d900d-5d16-424e-9be2-5823b54ebb14 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:20.067408 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220af91-6f38-46a1-a958-60e44a81cecc None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:20.067685 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220af91-6f38-46a1-a958-60e44a81cecc None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:20.067972 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220af91-6f38-46a1-a958-60e44a81cecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:20.086319 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-8220af91-6f38-46a1-a958-60e44a81cecc None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:20.087543 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-8220af91-6f38-46a1-a958-60e44a81cecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:20.102326 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:20.113972 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896caa64-dce9-4571-b625-9a232c481117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:20.115203 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896caa64-dce9-4571-b625-9a232c481117 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:20.116085 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896caa64-dce9-4571-b625-9a232c481117 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:20.116593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896caa64-dce9-4571-b625-9a232c481117 None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:20.117675 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896caa64-dce9-4571-b625-9a232c481117 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:20.118441 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896caa64-dce9-4571-b625-9a232c481117 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:20.120727 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896caa64-dce9-4571-b625-9a232c481117 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:20.140858 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 394/801] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:20.140858 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896caa64-dce9-4571-b625-9a232c481117 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:20.170643 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:20.177115 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 408/802] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:20:20.178339 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8220af91-6f38-46a1-a958-60e44a81cecc None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:20.309698 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896caa64-dce9-4571-b625-9a232c481117 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:20.359889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8220af91-6f38-46a1-a958-60e44a81cecc None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:20.368232 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cdcf16-093f-4ac1-a223-30744868860c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:20.369254 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cdcf16-093f-4ac1-a223-30744868860c None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:20.370402 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cdcf16-093f-4ac1-a223-30744868860c None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:20.370737 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cdcf16-093f-4ac1-a223-30744868860c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:20.409890 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84bec1-3494-44c6-b1ed-738daadfe2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:20.411225 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84bec1-3494-44c6-b1ed-738daadfe2ee None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:20.411718 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84bec1-3494-44c6-b1ed-738daadfe2ee None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:20.413112 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84bec1-3494-44c6-b1ed-738daadfe2ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:20.423167 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d84bec1-3494-44c6-b1ed-738daadfe2ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:20.425656 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d84bec1-3494-44c6-b1ed-738daadfe2ee None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:20.442072 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-3d84bec1-3494-44c6-b1ed-738daadfe2ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:20.442072 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:20.442072 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:20.442072 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:20.442072 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:20.442072 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:20.442072 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:20.442072 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:20.442072 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:20.442072 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:20.442072 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:20.442072 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:20.442072 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:20:20.442072 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:20:20.442072 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:20:20.442978 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:20:20.442978 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:20.442978 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:20.442978 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:20.442978 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:20.442978 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:20.442978 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:20.442978 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:20.442978 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:20.442978 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:20.442978 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:20.442978 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:20.442978 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:20.442978 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:20.442978 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:20.442978 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:20.442978 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:20.443876 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:20:20.443876 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:20:20.443876 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:20:20.443876 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:20:20.443876 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:20:20.443876 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:20:20.443876 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:20.443876 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:20.446848 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 409/803] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:20:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:20.447652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84bec1-3494-44c6-b1ed-738daadfe2ee None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:20.449358 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cdcf16-093f-4ac1-a223-30744868860c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:20.452762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cdcf16-093f-4ac1-a223-30744868860c None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:20.459747 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:20.482494 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:20.554239 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 395/804] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:20:20 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 10:20:20.555075 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cdcf16-093f-4ac1-a223-30744868860c None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:20.562798 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84bec1-3494-44c6-b1ed-738daadfe2ee None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:20.597743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da518ef5-7d38-4c4f-b14e-1bf49644b020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:20.598385 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da518ef5-7d38-4c4f-b14e-1bf49644b020 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:20.598662 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da518ef5-7d38-4c4f-b14e-1bf49644b020 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:20.598933 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da518ef5-7d38-4c4f-b14e-1bf49644b020 None admin] PATH_INFO: `/v3/services/network` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:20.599506 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da518ef5-7d38-4c4f-b14e-1bf49644b020 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:20.600846 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da518ef5-7d38-4c4f-b14e-1bf49644b020 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:20.609957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-da518ef5-7d38-4c4f-b14e-1bf49644b020 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 10:20:20.609957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:20.609957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 10:20:20.609957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 10:20:20.609957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 10:20:20.609957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 10:20:20.609957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 10:20:20.609957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:20.609957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 10:20:20.609957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:20.609957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 10:20:20.609957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:20.609957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:20.609957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:20.609957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:20.609957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:20.609957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:20.611037 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:20.611037 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:20.611037 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:20.611037 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:20.611037 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:20.611037 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:20.611037 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 10:20:20.611037 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 10:20:20.611037 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 10:20:20.611037 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 10:20:20.611037 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:20.611037 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:20.611037 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:20.611037 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:20.611037 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:20.611037 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:20.611880 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:20.611880 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:20.611880 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:20.611880 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:20.611880 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:20.611880 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:20.611880 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:20.611880 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:20.611880 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:20.611880 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:20.611880 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 10:20:20.611880 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:20.611880 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 10:20:20.611880 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:20.611880 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 410/805] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:20:20 2022] GET /identity/v3/services/network => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:20.612661 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da518ef5-7d38-4c4f-b14e-1bf49644b020 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:20.653026 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cdcf16-093f-4ac1-a223-30744868860c None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:20.678032 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:20.679687 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:20.687926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acbcb35-0c06-4027-b52f-229efcc47092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:20.688537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbcb35-0c06-4027-b52f-229efcc47092 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:20.688998 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbcb35-0c06-4027-b52f-229efcc47092 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:20.689110 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbcb35-0c06-4027-b52f-229efcc47092 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:20.698794 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acbcb35-0c06-4027-b52f-229efcc47092 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:20:20.703570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-1acbcb35-0c06-4027-b52f-229efcc47092 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:20:20.703570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:20.703570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:20.703570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:20.703570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:20.703570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:20.703570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:20.703570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:20.703570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:20.703570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:20.703570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:20.703570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:20.703570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:20:20.703570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:20:20.703570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:20:20.703570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:20:20.704661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:20:20.704661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:20:20.704661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:20:20.704661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:20:20.704661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:20.704661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:20.704661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:20.704661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:20.704661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:20.704661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:20.704661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:20.704661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:20.704661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:20.704661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:20.704661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:20.704661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:20.705495 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:20.705495 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:20.705495 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:20.705495 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:20.705495 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:20:20.705495 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:20:20.705495 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:20:20.705495 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:20:20.705495 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:20:20.705495 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:20:20.705495 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:20:20.705495 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:20.705495 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 396/806] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 10:20:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:20.706896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281b007-7064-4ab6-8d8c-0dcc6e129a53 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:20.706896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281b007-7064-4ab6-8d8c-0dcc6e129a53 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:20.706896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281b007-7064-4ab6-8d8c-0dcc6e129a53 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:20.708204 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 397/807] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:20.712328 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7281b007-7064-4ab6-8d8c-0dcc6e129a53 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:20.712444 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da518ef5-7d38-4c4f-b14e-1bf49644b020 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:20.743967 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dba863-a587-4b79-8c9b-8424c8054a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:20.744566 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dba863-a587-4b79-8c9b-8424c8054a79 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:20.744839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dba863-a587-4b79-8c9b-8424c8054a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:20.745104 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dba863-a587-4b79-8c9b-8424c8054a79 None admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:20.745778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dba863-a587-4b79-8c9b-8424c8054a79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:20.746105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dba863-a587-4b79-8c9b-8424c8054a79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:20.748053 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dba863-a587-4b79-8c9b-8424c8054a79 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:20.759609 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 411/808] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:20:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:20.760382 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dba863-a587-4b79-8c9b-8424c8054a79 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:20.820404 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7281b007-7064-4ab6-8d8c-0dcc6e129a53 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:20.863195 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66887f06-8022-4e5a-874a-05a173bff557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:20.863739 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66887f06-8022-4e5a-874a-05a173bff557 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:20.864017 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66887f06-8022-4e5a-874a-05a173bff557 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:20.864267 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dba863-a587-4b79-8c9b-8424c8054a79 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:20.864366 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66887f06-8022-4e5a-874a-05a173bff557 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:20.876386 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66887f06-8022-4e5a-874a-05a173bff557 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:20.878550 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66887f06-8022-4e5a-874a-05a173bff557 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:20.893249 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:20.895734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-66887f06-8022-4e5a-874a-05a173bff557 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:20.895734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:20.895734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:20.895734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:20.895734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:20.895734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:20.895734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:20.895734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:20.895734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:20.895734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:20.895734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:20.895734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:20.895734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:20.895734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:20.895734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:20.895734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:20.896653 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:20.896653 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:20.896653 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:20.896653 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:20.896653 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:20.896653 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:20.896653 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:20.896653 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:20.896653 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:20.896653 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:20.896653 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:20.896653 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:20.896653 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:20.896653 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:20.896653 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:20.896653 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:20.896653 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:20.897629 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:20.897629 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:20.897629 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:20.897629 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 398/809] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:20.897629 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66887f06-8022-4e5a-874a-05a173bff557 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:20.899870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73092bd-3678-49e7-a27a-0f14c86966da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:20.900475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73092bd-3678-49e7-a27a-0f14c86966da None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:20.900749 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73092bd-3678-49e7-a27a-0f14c86966da None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:20.901023 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73092bd-3678-49e7-a27a-0f14c86966da None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:20.901642 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73092bd-3678-49e7-a27a-0f14c86966da None admin] RBAC: Adding query filter params (name=network) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:20.901950 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73092bd-3678-49e7-a27a-0f14c86966da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:20.903764 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73092bd-3678-49e7-a27a-0f14c86966da None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:20.908465 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 412/810] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Sep 26 10:20:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:20.909577 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce4de1-1d3d-413e-8753-bbe5a2e7bf9d None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:20.909858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce4de1-1d3d-413e-8753-bbe5a2e7bf9d None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:20.910156 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce4de1-1d3d-413e-8753-bbe5a2e7bf9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:20.927538 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-5cce4de1-1d3d-413e-8753-bbe5a2e7bf9d None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:20.928719 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-5cce4de1-1d3d-413e-8753-bbe5a2e7bf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:20.936883 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:20.993619 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 413/811] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:20:20.994452 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce4de1-1d3d-413e-8753-bbe5a2e7bf9d None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:20.995915 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66887f06-8022-4e5a-874a-05a173bff557 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:21.029486 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026e3b1-8eee-407f-97a9-54090f817253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:21.030113 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026e3b1-8eee-407f-97a9-54090f817253 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:21.030389 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026e3b1-8eee-407f-97a9-54090f817253 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:21.030656 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026e3b1-8eee-407f-97a9-54090f817253 None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:21.031306 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0026e3b1-8eee-407f-97a9-54090f817253 None admin] RBAC: Adding query filter params (name=service) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:21.031603 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0026e3b1-8eee-407f-97a9-54090f817253 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:21.032953 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0026e3b1-8eee-407f-97a9-54090f817253 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:21.046144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 399/812] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 10:20:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:21.046989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026e3b1-8eee-407f-97a9-54090f817253 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:21.101197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce4de1-1d3d-413e-8753-bbe5a2e7bf9d None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:21.126866 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:21.137367 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2e314-6c9e-454d-bb73-0008c4099038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:21.138759 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2e314-6c9e-454d-bb73-0008c4099038 None admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:21.139202 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2e314-6c9e-454d-bb73-0008c4099038 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:21.139476 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2e314-6c9e-454d-bb73-0008c4099038 None admin] PATH_INFO: `/v3/projects/cff3a6011f4c4893bddffdad3d69f3df/users/5e6be34fc93a4551a83e5b8380188a6b/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:21.141406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea2e314-6c9e-454d-bb73-0008c4099038 None admin] RBAC: Authorizing `identity:create_grant(project_id=cff3a6011f4c4893bddffdad3d69f3df, user_id=5e6be34fc93a4551a83e5b8380188a6b, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:21.142881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea2e314-6c9e-454d-bb73-0008c4099038 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:21.151913 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:21.155643 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026e3b1-8eee-407f-97a9-54090f817253 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:21.166342 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 414/813] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 10:20:20 2022] PUT /identity/v3/projects/cff3a6011f4c4893bddffdad3d69f3df/users/5e6be34fc93a4551a83e5b8380188a6b/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:20:21.166342 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2e314-6c9e-454d-bb73-0008c4099038 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:21.199308 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161bce15-064e-47be-ba27-4d0e2b20a8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:21.200903 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161bce15-064e-47be-ba27-4d0e2b20a8d7 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:21.201254 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161bce15-064e-47be-ba27-4d0e2b20a8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:21.201590 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161bce15-064e-47be-ba27-4d0e2b20a8d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:21.202570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161bce15-064e-47be-ba27-4d0e2b20a8d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:21.202939 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161bce15-064e-47be-ba27-4d0e2b20a8d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:21.204752 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161bce15-064e-47be-ba27-4d0e2b20a8d7 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:21.227394 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 400/814] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:21.229087 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161bce15-064e-47be-ba27-4d0e2b20a8d7 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:21.312757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2e314-6c9e-454d-bb73-0008c4099038 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:21.347600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161bce15-064e-47be-ba27-4d0e2b20a8d7 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:21.359043 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17ba5b-34e8-4ce6-8283-1b4daba82318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:21.359861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17ba5b-34e8-4ce6-8283-1b4daba82318 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:21.360201 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17ba5b-34e8-4ce6-8283-1b4daba82318 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:21.360413 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17ba5b-34e8-4ce6-8283-1b4daba82318 None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:21.361044 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e17ba5b-34e8-4ce6-8283-1b4daba82318 None admin] RBAC: Adding query filter params (type=network) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:21.361345 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e17ba5b-34e8-4ce6-8283-1b4daba82318 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:21.362737 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e17ba5b-34e8-4ce6-8283-1b4daba82318 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:21.367984 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 415/815] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Sep 26 10:20:21 2022] GET /identity/v3/services?type=network => generated 350 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:21.369190 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17ba5b-34e8-4ce6-8283-1b4daba82318 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:21.387301 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a8f01-0bbd-487c-82ce-bd397d8d004f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:21.388490 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a8f01-0bbd-487c-82ce-bd397d8d004f None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:21.388776 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a8f01-0bbd-487c-82ce-bd397d8d004f None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:21.389177 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a8f01-0bbd-487c-82ce-bd397d8d004f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:21.400080 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a8f01-0bbd-487c-82ce-bd397d8d004f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:21.401773 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a8f01-0bbd-487c-82ce-bd397d8d004f None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:21.417421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-615a8f01-0bbd-487c-82ce-bd397d8d004f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:21.417421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:21.417421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:21.417421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:21.417421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:21.417421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:21.417421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:21.417421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:21.417421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:21.417421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:21.417421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:21.417421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:21.417421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:21.417421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:21.417421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:21.417421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:21.418577 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:21.418577 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:21.418577 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:21.418577 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:21.418577 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:21.418577 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:21.418577 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:21.418577 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:21.418577 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:21.418577 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:21.418577 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:21.418577 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:21.418577 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:21.418577 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:21.418577 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:21.418577 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:21.418577 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:21.419973 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:21.419973 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:21.419973 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:21.419973 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 401/816] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:21.419973 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a8f01-0bbd-487c-82ce-bd397d8d004f None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:21.426792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:21.439398 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:21.482963 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17ba5b-34e8-4ce6-8283-1b4daba82318 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:21.519221 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab4868-d935-490f-9ec9-67e719d9c26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:21.519881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab4868-d935-490f-9ec9-67e719d9c26b None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:21.520159 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab4868-d935-490f-9ec9-67e719d9c26b None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:21.520432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab4868-d935-490f-9ec9-67e719d9c26b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:21.527111 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a8f01-0bbd-487c-82ce-bd397d8d004f None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:21.527255 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab4868-d935-490f-9ec9-67e719d9c26b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:21.529082 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab4868-d935-490f-9ec9-67e719d9c26b None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:21.540556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-4aab4868-d935-490f-9ec9-67e719d9c26b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:21.540556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:21.540556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:21.540556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:21.540556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:21.540556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:21.540556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:21.540556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:21.540556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:21.540556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:21.540556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:21.540556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:21.540556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:20:21.540556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:20:21.540556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:20:21.541655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:20:21.541655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:21.541655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:21.541655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:21.541655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:21.541655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:21.541655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:21.541655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:21.541655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:21.541655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:21.541655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:21.541655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:21.541655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:21.541655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:21.541655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:21.541655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:21.541655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:21.542761 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:20:21.542761 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:20:21.542761 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:20:21.542761 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:20:21.542761 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:20:21.542761 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:20:21.542761 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:21.542761 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:21.542761 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 416/817] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:20:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:21.542761 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab4868-d935-490f-9ec9-67e719d9c26b None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:21.567545 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553160c2-02ed-4a33-bed4-a9ea9c5311f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:21.568180 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553160c2-02ed-4a33-bed4-a9ea9c5311f5 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:21.568464 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553160c2-02ed-4a33-bed4-a9ea9c5311f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:21.568740 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553160c2-02ed-4a33-bed4-a9ea9c5311f5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:21.569318 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553160c2-02ed-4a33-bed4-a9ea9c5311f5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:21.570739 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553160c2-02ed-4a33-bed4-a9ea9c5311f5 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:21.579615 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-553160c2-02ed-4a33-bed4-a9ea9c5311f5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 10:20:21.579615 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:21.579615 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 10:20:21.579615 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 10:20:21.579615 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 10:20:21.579615 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 10:20:21.579615 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 10:20:21.579615 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:21.579615 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 10:20:21.579615 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:21.579615 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 10:20:21.579615 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:21.579615 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:21.579615 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:21.579615 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:21.579615 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:21.579615 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:21.580600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:21.580600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:21.580600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:21.580600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:21.580600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:21.580600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:21.580600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 10:20:21.580600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 10:20:21.580600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 10:20:21.580600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 10:20:21.580600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:21.580600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:21.580600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:21.580600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:21.580600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:21.580600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:21.583074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:21.583074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:21.583074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:21.583074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:21.583074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:21.583074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:21.583074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:21.583074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:21.583074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:21.583074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:21.583074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 10:20:21.583074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:21.583074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 10:20:21.583074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:21.583074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 402/818] 10.222.0.49 () {56 vars in 1129 bytes} [Mon Sep 26 10:20:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:21.583898 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553160c2-02ed-4a33-bed4-a9ea9c5311f5 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:21.655958 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab4868-d935-490f-9ec9-67e719d9c26b None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:21.667744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:21.671820 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:21.694326 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75230b40-4a7a-4ca0-884b-2fb74e00715a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:21.694946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75230b40-4a7a-4ca0-884b-2fb74e00715a None admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:21.695189 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75230b40-4a7a-4ca0-884b-2fb74e00715a None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:21.695459 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75230b40-4a7a-4ca0-884b-2fb74e00715a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:21.695815 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553160c2-02ed-4a33-bed4-a9ea9c5311f5 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:21.696118 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75230b40-4a7a-4ca0-884b-2fb74e00715a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:21.697470 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75230b40-4a7a-4ca0-884b-2fb74e00715a None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:21.722060 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 417/819] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:20:21 2022] POST /identity/v3/endpoints => generated 337 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:20:21.722862 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75230b40-4a7a-4ca0-884b-2fb74e00715a None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:21.759759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da424be-7445-4f0e-98b9-b9c19413c5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:21.760384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da424be-7445-4f0e-98b9-b9c19413c5de None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:21.760663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da424be-7445-4f0e-98b9-b9c19413c5de None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:21.760933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da424be-7445-4f0e-98b9-b9c19413c5de None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:21.761620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da424be-7445-4f0e-98b9-b9c19413c5de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:21.761918 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da424be-7445-4f0e-98b9-b9c19413c5de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:21.765411 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da424be-7445-4f0e-98b9-b9c19413c5de None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:21.782078 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 403/820] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:21.782078 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da424be-7445-4f0e-98b9-b9c19413c5de None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:21.862861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75230b40-4a7a-4ca0-884b-2fb74e00715a None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:21.913382 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984cb49a-88b0-4d48-8977-c196ce1e8d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:21.914782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984cb49a-88b0-4d48-8977-c196ce1e8d16 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:21.914782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984cb49a-88b0-4d48-8977-c196ce1e8d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:21.915079 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984cb49a-88b0-4d48-8977-c196ce1e8d16 None admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:21.915910 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984cb49a-88b0-4d48-8977-c196ce1e8d16 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:21.916218 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984cb49a-88b0-4d48-8977-c196ce1e8d16 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:21.918186 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984cb49a-88b0-4d48-8977-c196ce1e8d16 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:21.933204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 418/821] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:20:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:21.934265 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984cb49a-88b0-4d48-8977-c196ce1e8d16 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:21.939147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da424be-7445-4f0e-98b9-b9c19413c5de None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:21.971469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:21.984405 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:21.994392 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94315d76-8ae1-429a-89e8-9aeb28b7a9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:21.995010 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94315d76-8ae1-429a-89e8-9aeb28b7a9b5 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:21.995282 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94315d76-8ae1-429a-89e8-9aeb28b7a9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:21.995551 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94315d76-8ae1-429a-89e8-9aeb28b7a9b5 None admin] PATH_INFO: `/v3/services` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:21.996735 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94315d76-8ae1-429a-89e8-9aeb28b7a9b5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:21.996735 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94315d76-8ae1-429a-89e8-9aeb28b7a9b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:21.998012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94315d76-8ae1-429a-89e8-9aeb28b7a9b5 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:22.007464 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 404/822] 10.222.0.49 () {56 vars in 1124 bytes} [Mon Sep 26 10:20:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:22.008389 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94315d76-8ae1-429a-89e8-9aeb28b7a9b5 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:22.067611 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984cb49a-88b0-4d48-8977-c196ce1e8d16 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:22.107478 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60818ab2-e328-43ca-966a-1bfc5c9aeff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:22.108106 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60818ab2-e328-43ca-966a-1bfc5c9aeff2 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:22.108380 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60818ab2-e328-43ca-966a-1bfc5c9aeff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:22.108649 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60818ab2-e328-43ca-966a-1bfc5c9aeff2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:22.116050 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60818ab2-e328-43ca-966a-1bfc5c9aeff2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:22.117470 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60818ab2-e328-43ca-966a-1bfc5c9aeff2 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:22.117668 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94315d76-8ae1-429a-89e8-9aeb28b7a9b5 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:22.130278 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-60818ab2-e328-43ca-966a-1bfc5c9aeff2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 10:20:22.130278 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:22.130278 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:22.130278 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:22.130278 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:22.130278 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:22.130278 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:22.130278 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:22.130278 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:22.130278 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:22.130278 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:22.130278 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:22.130278 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:20:22.130278 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:20:22.130278 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:20:22.130278 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:20:22.131397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:22.131397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:22.131397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:20:22.131397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:22.131397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:20:22.131397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:22.131397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:22.131397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:22.131397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:22.131397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:22.131397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:22.131397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:22.131397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:22.131397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:22.131397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:22.131397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:22.131397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:22.131397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:22.132556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:22.132556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:22.132556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:20:22.132556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:20:22.132556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:20:22.132556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:20:22.132556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:20:22.132556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:20:22.132556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 10:20:22.132556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:22.136199 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 419/823] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Sep 26 10:20:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:22.137013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60818ab2-e328-43ca-966a-1bfc5c9aeff2 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:22.160436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5bcc6-7ab6-4d2e-a052-fdb64e45ccf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:22.161326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5bcc6-7ab6-4d2e-a052-fdb64e45ccf5 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:22.161893 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5bcc6-7ab6-4d2e-a052-fdb64e45ccf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:22.162332 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5bcc6-7ab6-4d2e-a052-fdb64e45ccf5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:22.163808 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5bcc6-7ab6-4d2e-a052-fdb64e45ccf5 None admin] RBAC: Adding query filter params (role.id=532b37d9076c45618cf352aafb85f35e, scope.project.id=cff3a6011f4c4893bddffdad3d69f3df, user.id=cee41014d5d043888c6b399710bccab4) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:22.164261 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5bcc6-7ab6-4d2e-a052-fdb64e45ccf5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:22.166565 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5bcc6-7ab6-4d2e-a052-fdb64e45ccf5 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:22.186562 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 405/824] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:20:22 2022] GET /identity/v3/role_assignments?user.id=cee41014d5d043888c6b399710bccab4&scope.project.id=cff3a6011f4c4893bddffdad3d69f3df&role.id=532b37d9076c45618cf352aafb85f35e => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:22.186562 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5bcc6-7ab6-4d2e-a052-fdb64e45ccf5 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:22.249127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:22.255957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:22.278756 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60818ab2-e328-43ca-966a-1bfc5c9aeff2 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:22.294861 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5bcc6-7ab6-4d2e-a052-fdb64e45ccf5 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:22.333334 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a2f2d-e3fb-412e-8756-3db0ee1a7f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:22.333548 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5ed4f-b30d-45c4-8f66-f6d3d9a29083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:22.334194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a2f2d-e3fb-412e-8756-3db0ee1a7f17 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:22.334295 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5ed4f-b30d-45c4-8f66-f6d3d9a29083 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:22.334462 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5ed4f-b30d-45c4-8f66-f6d3d9a29083 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:22.334594 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a2f2d-e3fb-412e-8756-3db0ee1a7f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:22.334734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5ed4f-b30d-45c4-8f66-f6d3d9a29083 None admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:22.335043 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a2f2d-e3fb-412e-8756-3db0ee1a7f17 None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:22.335459 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea5ed4f-b30d-45c4-8f66-f6d3d9a29083 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:22.335756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea5ed4f-b30d-45c4-8f66-f6d3d9a29083 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:22.335850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a2f2d-e3fb-412e-8756-3db0ee1a7f17 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:22.336178 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a2f2d-e3fb-412e-8756-3db0ee1a7f17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:22.337170 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea5ed4f-b30d-45c4-8f66-f6d3d9a29083 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:22.337677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a2f2d-e3fb-412e-8756-3db0ee1a7f17 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:22.345323 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 420/825] 10.222.0.49 () {56 vars in 1124 bytes} [Mon Sep 26 10:20:22 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:22.351363 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a2f2d-e3fb-412e-8756-3db0ee1a7f17 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:22.365794 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 406/826] 10.222.0.49 () {56 vars in 1149 bytes} [Mon Sep 26 10:20:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:22.367235 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5ed4f-b30d-45c4-8f66-f6d3d9a29083 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:22.474522 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a2f2d-e3fb-412e-8756-3db0ee1a7f17 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:22.495112 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5ed4f-b30d-45c4-8f66-f6d3d9a29083 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:22.517442 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:22.518603 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f494f-9418-4b7d-8617-fcbb9371e3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:22.519337 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f494f-9418-4b7d-8617-fcbb9371e3eb None admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:22.519720 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f494f-9418-4b7d-8617-fcbb9371e3eb None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:22.520082 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f494f-9418-4b7d-8617-fcbb9371e3eb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:22.520854 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0f494f-9418-4b7d-8617-fcbb9371e3eb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:22.522971 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0f494f-9418-4b7d-8617-fcbb9371e3eb None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:22.542830 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078030d-8155-47c6-96c7-3466e816bc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:22.543514 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078030d-8155-47c6-96c7-3466e816bc4b None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:22.543739 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078030d-8155-47c6-96c7-3466e816bc4b None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:22.544513 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078030d-8155-47c6-96c7-3466e816bc4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:22.552079 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 421/827] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:20:22 2022] POST /identity/v3/endpoints => generated 346 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:20:22.563511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078030d-8155-47c6-96c7-3466e816bc4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:22.564989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078030d-8155-47c6-96c7-3466e816bc4b None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:22.583313 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-5078030d-8155-47c6-96c7-3466e816bc4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:22.583313 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:22.583313 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:22.583313 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:22.583313 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:22.583313 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:22.583313 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:22.583313 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:22.583313 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:22.583313 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:22.583313 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:22.583313 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:22.583313 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:22.583313 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:22.583313 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:22.583313 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:22.584352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:22.584352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:22.584352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:22.584352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:22.584352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:22.584352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:22.584352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:22.584352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:22.584352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:22.584352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:22.584352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:22.584352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:22.584352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:22.584352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:22.584352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:22.584352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:22.584352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:22.585262 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:22.585262 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:22.585262 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:22.585262 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 407/828] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:20:22.590382 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f494f-9418-4b7d-8617-fcbb9371e3eb None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:22.609561 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:22.717190 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f494f-9418-4b7d-8617-fcbb9371e3eb None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:22.760475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbaa6b3-0595-431c-b342-333479ca6a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:22.761462 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbaa6b3-0595-431c-b342-333479ca6a62 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:22.761936 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbaa6b3-0595-431c-b342-333479ca6a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:22.762424 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbaa6b3-0595-431c-b342-333479ca6a62 None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:22.763321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbaa6b3-0595-431c-b342-333479ca6a62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:22.763624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbaa6b3-0595-431c-b342-333479ca6a62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:22.765112 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbaa6b3-0595-431c-b342-333479ca6a62 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:22.777512 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 422/829] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:22.783528 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078030d-8155-47c6-96c7-3466e816bc4b None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:22.898576 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078030d-8155-47c6-96c7-3466e816bc4b None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:22.932707 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065873d-dc5d-45b8-83d7-2dab104d5ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:22.933439 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065873d-dc5d-45b8-83d7-2dab104d5ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:22.933719 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065873d-dc5d-45b8-83d7-2dab104d5ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:22.933988 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065873d-dc5d-45b8-83d7-2dab104d5ca3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:22.941383 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4065873d-dc5d-45b8-83d7-2dab104d5ca3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:22.942899 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4065873d-dc5d-45b8-83d7-2dab104d5ca3 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:22.954777 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-4065873d-dc5d-45b8-83d7-2dab104d5ca3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:22.954777 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:22.954777 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:22.954777 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:22.954777 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:22.954777 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:22.954777 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:22.954777 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:22.954777 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:22.954777 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:22.954777 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:22.954777 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:22.954777 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:20:22.954777 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:20:22.954777 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:20:22.955749 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:20:22.955749 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:22.955749 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:22.955749 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:22.955749 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:22.955749 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:22.955749 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:22.955749 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:22.955749 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:22.955749 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:22.955749 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:22.955749 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:22.955749 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:22.955749 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:22.955749 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:22.955749 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:22.955749 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:22.956911 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:20:22.956911 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:20:22.956911 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:20:22.956911 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:20:22.956911 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:20:22.956911 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:20:22.956911 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:22.956911 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:22.960843 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 408/830] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:20:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:22.962228 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbaa6b3-0595-431c-b342-333479ca6a62 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:23.028564 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:23.070434 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbaa6b3-0595-431c-b342-333479ca6a62 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:23.106363 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d5820-bc06-41e9-938b-94a3e61703e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:23.107051 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d5820-bc06-41e9-938b-94a3e61703e3 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:23.107331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d5820-bc06-41e9-938b-94a3e61703e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:23.107603 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d5820-bc06-41e9-938b-94a3e61703e3 None admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:23.108292 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d5820-bc06-41e9-938b-94a3e61703e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:23.108592 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d5820-bc06-41e9-938b-94a3e61703e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:23.110086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d5820-bc06-41e9-938b-94a3e61703e3 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:23.128288 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 423/831] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:20:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:23.129597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065873d-dc5d-45b8-83d7-2dab104d5ca3 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:23.161712 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:23.235877 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065873d-dc5d-45b8-83d7-2dab104d5ca3 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:23.274476 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bdb1e8-efaf-442d-8aae-9cefe93426ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:23.275945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdb1e8-efaf-442d-8aae-9cefe93426ea None admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:23.276219 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdb1e8-efaf-442d-8aae-9cefe93426ea None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:23.276488 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdb1e8-efaf-442d-8aae-9cefe93426ea None admin] PATH_INFO: `/v3/projects/cff3a6011f4c4893bddffdad3d69f3df/users/9158557f1a2947e2815dc8d586d8b0c3/roles/d36e58344b604bcda1cee49065e8497e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:23.292950 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bdb1e8-efaf-442d-8aae-9cefe93426ea None admin] RBAC: Authorizing `identity:create_grant(project_id=cff3a6011f4c4893bddffdad3d69f3df, user_id=9158557f1a2947e2815dc8d586d8b0c3, role_id=d36e58344b604bcda1cee49065e8497e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:23.294859 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bdb1e8-efaf-442d-8aae-9cefe93426ea None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:23.308758 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 409/832] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 10:20:23 2022] PUT /identity/v3/projects/cff3a6011f4c4893bddffdad3d69f3df/users/9158557f1a2947e2815dc8d586d8b0c3/roles/d36e58344b604bcda1cee49065e8497e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:20:23.376679 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df0d1c-18af-495e-b4a9-2189cdfd058d None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:23.376960 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df0d1c-18af-495e-b4a9-2189cdfd058d None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:23.377232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df0d1c-18af-495e-b4a9-2189cdfd058d None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:23.378952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 424/833] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:23.384251 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23400e-ac36-4e57-aa5a-cad56cf4816c None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:23.384789 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23400e-ac36-4e57-aa5a-cad56cf4816c None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:23.385249 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23400e-ac36-4e57-aa5a-cad56cf4816c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:23.406180 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-ea23400e-ac36-4e57-aa5a-cad56cf4816c None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:23.407845 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-ea23400e-ac36-4e57-aa5a-cad56cf4816c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:23.488319 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 410/834] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:23 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:20:23.757948 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df0d1c-18af-495e-b4a9-2189cdfd058d None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:23.867964 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df0d1c-18af-495e-b4a9-2189cdfd058d None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:23.891448 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:23.904257 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd7ba7-93fe-46b1-a5e1-001e319c4a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:23.904886 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd7ba7-93fe-46b1-a5e1-001e319c4a58 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:23.905163 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd7ba7-93fe-46b1-a5e1-001e319c4a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:23.905435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd7ba7-93fe-46b1-a5e1-001e319c4a58 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:23.916342 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dd7ba7-93fe-46b1-a5e1-001e319c4a58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:20:23.921858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-68dd7ba7-93fe-46b1-a5e1-001e319c4a58 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:20:23.921858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:23.921858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:23.921858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:23.921858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:23.921858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:23.921858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:23.921858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:23.921858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:23.921858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:23.921858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:23.921858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:23.921858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:20:23.921858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:20:23.921858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:20:23.921858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:20:23.922792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:20:23.922792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:20:23.922792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:20:23.922792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:20:23.922792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:23.922792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:23.922792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:23.922792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:23.922792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:23.922792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:23.922792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:23.922792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:23.922792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:23.922792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:23.922792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:23.922792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:23.923851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:23.923851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:23.923851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:23.923851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:23.923851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:20:23.923851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:20:23.923851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:20:23.923851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:20:23.923851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:20:23.923851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:20:23.923851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:20:23.923851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:23.923851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 425/835] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:20:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:23.929259 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23400e-ac36-4e57-aa5a-cad56cf4816c None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:23.940829 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:24.036789 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23400e-ac36-4e57-aa5a-cad56cf4816c None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:24.071608 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c2095f-01cf-47d8-b752-a65aad9715c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:24.072215 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c2095f-01cf-47d8-b752-a65aad9715c0 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:24.072488 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c2095f-01cf-47d8-b752-a65aad9715c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:24.072756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c2095f-01cf-47d8-b752-a65aad9715c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:24.073408 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c2095f-01cf-47d8-b752-a65aad9715c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:24.073704 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c2095f-01cf-47d8-b752-a65aad9715c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:24.075096 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c2095f-01cf-47d8-b752-a65aad9715c0 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:24.084892 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 411/836] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:20:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:24.090654 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd7ba7-93fe-46b1-a5e1-001e319c4a58 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:24.199019 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd7ba7-93fe-46b1-a5e1-001e319c4a58 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:24.234640 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751a27c-1a5e-4c17-8b17-116faa612ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:24.235272 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751a27c-1a5e-4c17-8b17-116faa612ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:24.235550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751a27c-1a5e-4c17-8b17-116faa612ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:24.235821 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751a27c-1a5e-4c17-8b17-116faa612ba2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:24.247497 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3751a27c-1a5e-4c17-8b17-116faa612ba2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:24.248983 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3751a27c-1a5e-4c17-8b17-116faa612ba2 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:24.265768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-3751a27c-1a5e-4c17-8b17-116faa612ba2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:24.265768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:24.265768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:24.265768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:24.265768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:24.265768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:24.265768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:24.265768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:24.265768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:24.265768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:24.265768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:24.265768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:24.265768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:24.265768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:24.265768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:24.265768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:24.266802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:24.266802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:24.266802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:24.266802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:24.266802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:24.266802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:24.266802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:24.266802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:24.266802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:24.266802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:24.266802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:24.266802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:24.266802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:24.266802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:24.266802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:24.266802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:24.266802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:24.268110 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:24.268110 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:24.268110 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:24.268110 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 426/837] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:24.274285 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c2095f-01cf-47d8-b752-a65aad9715c0 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:24.347401 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:24.380358 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c2095f-01cf-47d8-b752-a65aad9715c0 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:24.418420 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa6f90-2229-48dd-b958-c8a0d95e6ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:24.419466 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa6f90-2229-48dd-b958-c8a0d95e6ccc None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:24.419969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa6f90-2229-48dd-b958-c8a0d95e6ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:24.420477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa6f90-2229-48dd-b958-c8a0d95e6ccc None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:24.421430 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa6f90-2229-48dd-b958-c8a0d95e6ccc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:24.421936 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa6f90-2229-48dd-b958-c8a0d95e6ccc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:24.423684 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa6f90-2229-48dd-b958-c8a0d95e6ccc None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:24.441554 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 412/838] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:24.447154 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751a27c-1a5e-4c17-8b17-116faa612ba2 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:24.483674 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:24.522687 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556b2b4-c8e9-473b-9e38-c37a9eb8df36 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:24.523019 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556b2b4-c8e9-473b-9e38-c37a9eb8df36 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:24.523228 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556b2b4-c8e9-473b-9e38-c37a9eb8df36 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:24.524828 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 413/839] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:20:24.529836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dae0aa-0ac7-4c62-8ecf-19b4ba01e281 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:24.530152 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dae0aa-0ac7-4c62-8ecf-19b4ba01e281 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:24.530428 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dae0aa-0ac7-4c62-8ecf-19b4ba01e281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:24.550908 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-16dae0aa-0ac7-4c62-8ecf-19b4ba01e281 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:24.552806 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-16dae0aa-0ac7-4c62-8ecf-19b4ba01e281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:24.563523 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751a27c-1a5e-4c17-8b17-116faa612ba2 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:24.601566 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc21cf0-8f7d-4a18-b587-c70b2584bf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:24.602148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc21cf0-8f7d-4a18-b587-c70b2584bf8d None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:24.602466 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc21cf0-8f7d-4a18-b587-c70b2584bf8d None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:24.602742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc21cf0-8f7d-4a18-b587-c70b2584bf8d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:24.610958 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc21cf0-8f7d-4a18-b587-c70b2584bf8d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:24.612458 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc21cf0-8f7d-4a18-b587-c70b2584bf8d None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:24.624900 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-2dc21cf0-8f7d-4a18-b587-c70b2584bf8d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 26 10:20:24.624900 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:24.624900 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:24.624900 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:24.624900 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:24.624900 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:24.624900 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:24.624900 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:24.624900 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:24.624900 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:24.624900 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:24.624900 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:24.624900 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:20:24.624900 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:20:24.624900 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:20:24.624900 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:20:24.625892 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:24.625892 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:24.625892 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:20:24.625892 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:24.625892 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:20:24.625892 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:24.625892 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:24.625892 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:24.625892 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:24.625892 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:24.625892 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:24.625892 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:24.625892 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:24.625892 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:24.625892 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:24.625892 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:24.625892 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:24.625892 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:24.627429 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:24.627429 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:24.627429 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:20:24.627429 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:20:24.627429 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:20:24.627429 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:20:24.627429 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:20:24.627429 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:20:24.627429 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 26 10:20:24.627429 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:24.630732 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 427/840] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:20:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:24.631665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc21cf0-8f7d-4a18-b587-c70b2584bf8d None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:24.649282 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 414/841] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:24 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 10:20:24.734155 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:24.742476 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc21cf0-8f7d-4a18-b587-c70b2584bf8d None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:24.777735 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9362185-b4c8-496f-8105-ae58be1ad1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:24.778547 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9362185-b4c8-496f-8105-ae58be1ad1dc None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:24.778734 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9362185-b4c8-496f-8105-ae58be1ad1dc None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:24.779005 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9362185-b4c8-496f-8105-ae58be1ad1dc None admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:24.779774 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9362185-b4c8-496f-8105-ae58be1ad1dc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:24.780077 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9362185-b4c8-496f-8105-ae58be1ad1dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:24.781483 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9362185-b4c8-496f-8105-ae58be1ad1dc None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:24.800989 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 428/842] 10.222.0.49 () {56 vars in 1135 bytes} [Mon Sep 26 10:20:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:24.811414 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dae0aa-0ac7-4c62-8ecf-19b4ba01e281 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:24.877949 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:24.924434 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dae0aa-0ac7-4c62-8ecf-19b4ba01e281 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:24.935895 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9362185-b4c8-496f-8105-ae58be1ad1dc None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:24.974746 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71a7a92-6340-4ddf-a58a-e4b9e51d922a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:24.975638 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a7a92-6340-4ddf-a58a-e4b9e51d922a None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:24.975915 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a7a92-6340-4ddf-a58a-e4b9e51d922a None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:24.976182 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a7a92-6340-4ddf-a58a-e4b9e51d922a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:25.053878 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71a7a92-6340-4ddf-a58a-e4b9e51d922a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:25.056302 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71a7a92-6340-4ddf-a58a-e4b9e51d922a None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:25.061536 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9362185-b4c8-496f-8105-ae58be1ad1dc None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:25.099097 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd9067-d4da-4774-bf97-3f038b9f21fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:25.099720 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd9067-d4da-4774-bf97-3f038b9f21fb None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:25.099995 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd9067-d4da-4774-bf97-3f038b9f21fb None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:25.100269 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd9067-d4da-4774-bf97-3f038b9f21fb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:25.100844 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd9067-d4da-4774-bf97-3f038b9f21fb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:25.102225 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd9067-d4da-4774-bf97-3f038b9f21fb None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:25.112071 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-aedd9067-d4da-4774-bf97-3f038b9f21fb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 10:20:25.112071 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:25.112071 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 10:20:25.112071 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 10:20:25.112071 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 10:20:25.112071 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 10:20:25.112071 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 10:20:25.112071 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:25.112071 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 10:20:25.112071 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:25.112071 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 10:20:25.112071 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:25.112071 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:25.112071 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:25.112071 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:25.112071 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:25.113215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:25.113215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:25.113215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:25.113215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:25.113215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:25.113215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:25.113215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:25.113215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 10:20:25.113215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 10:20:25.113215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 10:20:25.113215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 10:20:25.113215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:25.113215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:25.113215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:25.113215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:25.113215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:25.113215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:25.114244 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:25.114244 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:25.114244 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:25.114244 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:25.114244 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:25.114244 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:25.114244 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:25.114244 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:25.114244 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:25.114244 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:25.114244 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 10:20:25.114244 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:25.114244 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 10:20:25.114244 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:25.114244 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 429/843] 10.222.0.49 () {56 vars in 1132 bytes} [Mon Sep 26 10:20:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:20:25.115060 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eadc94-0dc3-49f8-9e85-12419094f249 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:25.115272 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eadc94-0dc3-49f8-9e85-12419094f249 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:25.115540 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eadc94-0dc3-49f8-9e85-12419094f249 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:25.118056 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 430/844] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:25.119593 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eadc94-0dc3-49f8-9e85-12419094f249 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:25.136352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:25.140148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:25.184103 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 415/845] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:20:24 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 10:20:25.186254 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b6e59-cc5b-4a69-baf3-46992fb3f933 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:25.186665 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b6e59-cc5b-4a69-baf3-46992fb3f933 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:25.187058 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b6e59-cc5b-4a69-baf3-46992fb3f933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:25.206477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-ac9b6e59-cc5b-4a69-baf3-46992fb3f933 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:25.207916 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-ac9b6e59-cc5b-4a69-baf3-46992fb3f933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:25.236442 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eadc94-0dc3-49f8-9e85-12419094f249 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:25.272994 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e579f9b-dd98-49a4-a933-3aaec081be80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:25.273809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e579f9b-dd98-49a4-a933-3aaec081be80 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:25.274227 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e579f9b-dd98-49a4-a933-3aaec081be80 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:25.274506 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e579f9b-dd98-49a4-a933-3aaec081be80 None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:25.275137 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e579f9b-dd98-49a4-a933-3aaec081be80 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:25.275437 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e579f9b-dd98-49a4-a933-3aaec081be80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:25.276793 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e579f9b-dd98-49a4-a933-3aaec081be80 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:25.290117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 416/846] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:20:25.290117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b6e59-cc5b-4a69-baf3-46992fb3f933 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:25.290353 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 431/847] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Sep 26 10:20:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:25.290353 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e579f9b-dd98-49a4-a933-3aaec081be80 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:25.381468 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:25.382741 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:25.392093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b6e59-cc5b-4a69-baf3-46992fb3f933 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:25.400047 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e579f9b-dd98-49a4-a933-3aaec081be80 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:25.426867 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd32cfa-0701-43b9-bdfe-b5433a3ac915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:25.427500 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd32cfa-0701-43b9-bdfe-b5433a3ac915 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:25.427777 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd32cfa-0701-43b9-bdfe-b5433a3ac915 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:25.428048 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd32cfa-0701-43b9-bdfe-b5433a3ac915 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:25.438688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83a2a4-0b9d-4d08-8669-371dc88baf21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:25.438974 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd32cfa-0701-43b9-bdfe-b5433a3ac915 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:25.439292 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83a2a4-0b9d-4d08-8669-371dc88baf21 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:25.439569 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83a2a4-0b9d-4d08-8669-371dc88baf21 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:25.439839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83a2a4-0b9d-4d08-8669-371dc88baf21 None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:25.440360 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd32cfa-0701-43b9-bdfe-b5433a3ac915 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:25.440467 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a83a2a4-0b9d-4d08-8669-371dc88baf21 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:25.440749 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a83a2a4-0b9d-4d08-8669-371dc88baf21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:25.442112 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a83a2a4-0b9d-4d08-8669-371dc88baf21 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:25.454588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 432/848] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Sep 26 10:20:25 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:25.456293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-7dd32cfa-0701-43b9-bdfe-b5433a3ac915 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:25.456293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:25.456293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:25.456293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:25.456293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:25.456293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:25.456293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:25.456293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:25.456293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:25.456293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:25.456293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:25.456293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:25.456293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:25.456293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:25.456293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:25.456293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:25.457520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:25.457520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:25.457520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:25.457520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:25.457520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:25.457520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:25.457520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:25.457520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:25.457520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:25.457520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:25.457520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:25.457520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:25.457520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:25.457520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:25.457520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:25.457520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:25.457520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:25.458437 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:25.458437 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:25.458437 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:25.462801 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 417/849] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:25.463930 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83a2a4-0b9d-4d08-8669-371dc88baf21 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:25.574632 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd32cfa-0701-43b9-bdfe-b5433a3ac915 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:25.583651 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83a2a4-0b9d-4d08-8669-371dc88baf21 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:25.621386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ae1be-f709-4e5e-a80c-1580f8f8fd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:25.622006 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ae1be-f709-4e5e-a80c-1580f8f8fd96 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:25.622882 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ae1be-f709-4e5e-a80c-1580f8f8fd96 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:25.623156 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ae1be-f709-4e5e-a80c-1580f8f8fd96 None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:25.623816 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ae1be-f709-4e5e-a80c-1580f8f8fd96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:25.624115 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ae1be-f709-4e5e-a80c-1580f8f8fd96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:25.625518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ae1be-f709-4e5e-a80c-1580f8f8fd96 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:25.638533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 433/850] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:25.639950 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53b22e-20a5-402b-8ee0-c0d8a4f615fe None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:25.640236 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53b22e-20a5-402b-8ee0-c0d8a4f615fe None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:25.640508 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53b22e-20a5-402b-8ee0-c0d8a4f615fe None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:25.641839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 434/851] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:25.644448 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53b22e-20a5-402b-8ee0-c0d8a4f615fe None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:25.650284 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:25.695862 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd32cfa-0701-43b9-bdfe-b5433a3ac915 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:25.730309 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05375b-b7f2-42be-9e31-5616706d18d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:25.730942 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05375b-b7f2-42be-9e31-5616706d18d2 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:25.731221 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05375b-b7f2-42be-9e31-5616706d18d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:25.732078 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05375b-b7f2-42be-9e31-5616706d18d2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:25.732618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe05375b-b7f2-42be-9e31-5616706d18d2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:25.733984 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe05375b-b7f2-42be-9e31-5616706d18d2 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:25.742896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-fe05375b-b7f2-42be-9e31-5616706d18d2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 10:20:25.742896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:25.742896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 10:20:25.742896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 10:20:25.742896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 10:20:25.742896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 10:20:25.742896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 10:20:25.742896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:25.742896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 10:20:25.742896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:25.742896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 10:20:25.742896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:25.742896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:25.742896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:25.742896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:25.742896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:25.742896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:25.743874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:25.743874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:25.743874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:25.743874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:25.743874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:25.743874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:25.743874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 10:20:25.743874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 10:20:25.743874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 10:20:25.743874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 10:20:25.743874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:25.743874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:25.743874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:25.743874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:25.743874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:25.743874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:25.744787 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:25.744787 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:25.744787 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:25.744787 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:25.744787 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:25.744787 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:25.744787 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:25.744787 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:25.744787 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:25.744787 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:25.744787 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 10:20:25.744787 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:25.744787 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 10:20:25.744787 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:25.744787 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 418/852] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 10:20:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:20:25.745568 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14561e9-d63f-4dad-a5b2-cd938f9fc329 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:25.745568 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14561e9-d63f-4dad-a5b2-cd938f9fc329 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:25.745695 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14561e9-d63f-4dad-a5b2-cd938f9fc329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:25.763814 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-b14561e9-d63f-4dad-a5b2-cd938f9fc329 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:25.765013 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-b14561e9-d63f-4dad-a5b2-cd938f9fc329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:25.766474 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53b22e-20a5-402b-8ee0-c0d8a4f615fe None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:25.767014 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:25.813962 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da6c95-bce0-4cb0-ac86-b3526de0746b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:25.815009 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da6c95-bce0-4cb0-ac86-b3526de0746b None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:25.815265 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da6c95-bce0-4cb0-ac86-b3526de0746b None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:25.815537 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da6c95-bce0-4cb0-ac86-b3526de0746b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:25.823133 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da6c95-bce0-4cb0-ac86-b3526de0746b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:25.824566 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da6c95-bce0-4cb0-ac86-b3526de0746b None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:25.836176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-e2da6c95-bce0-4cb0-ac86-b3526de0746b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:25.836176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:25.836176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:25.836176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:25.836176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:25.836176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:25.836176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:25.836176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:25.836176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:25.836176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:25.836176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:25.836176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:25.836176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:20:25.836176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:20:25.836176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:20:25.837086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:20:25.837086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:25.837086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:25.837086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:25.837086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:25.837086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:25.837086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:25.837086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:25.837086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:25.837086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:25.837086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:25.837086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:25.837086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:25.837086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:25.837086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:25.837086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:25.837086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:25.843000 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:20:25.843000 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:20:25.843000 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:20:25.843000 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:20:25.843000 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:20:25.843000 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:20:25.843000 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:25.843000 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:25.843000 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 435/853] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:20:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:25.843000 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da6c95-bce0-4cb0-ac86-b3526de0746b None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:25.843541 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 419/854] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:20:25.844240 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14561e9-d63f-4dad-a5b2-cd938f9fc329 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:25.921648 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:25.955513 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14561e9-d63f-4dad-a5b2-cd938f9fc329 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:25.967056 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da6c95-bce0-4cb0-ac86-b3526de0746b None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:25.995022 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148f8cc-33f7-404d-b51b-15462f268c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:25.995626 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148f8cc-33f7-404d-b51b-15462f268c53 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:25.995907 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148f8cc-33f7-404d-b51b-15462f268c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:25.996173 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148f8cc-33f7-404d-b51b-15462f268c53 None admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:25.996859 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148f8cc-33f7-404d-b51b-15462f268c53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:25.997188 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148f8cc-33f7-404d-b51b-15462f268c53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:25.998661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148f8cc-33f7-404d-b51b-15462f268c53 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:26.006155 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:26.008900 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f24ac-1104-4c05-a68e-92cf182b770d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:26.009713 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f24ac-1104-4c05-a68e-92cf182b770d None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:26.010200 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f24ac-1104-4c05-a68e-92cf182b770d None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:26.010612 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f24ac-1104-4c05-a68e-92cf182b770d None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:26.011392 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f24ac-1104-4c05-a68e-92cf182b770d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:26.012070 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 420/855] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:20:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:26.012913 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f24ac-1104-4c05-a68e-92cf182b770d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:26.022172 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f24ac-1104-4c05-a68e-92cf182b770d None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:26.022340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148f8cc-33f7-404d-b51b-15462f268c53 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:26.022448 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 436/856] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 10:20:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:26.030058 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f24ac-1104-4c05-a68e-92cf182b770d None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:26.138238 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f24ac-1104-4c05-a68e-92cf182b770d None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:26.156029 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148f8cc-33f7-404d-b51b-15462f268c53 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:26.167208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:26.177896 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c269513-92f9-41f7-989a-669cb0822e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:26.178466 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c269513-92f9-41f7-989a-669cb0822e0e None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:26.178731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c269513-92f9-41f7-989a-669cb0822e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:26.179004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c269513-92f9-41f7-989a-669cb0822e0e None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:26.179626 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c269513-92f9-41f7-989a-669cb0822e0e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:26.179923 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c269513-92f9-41f7-989a-669cb0822e0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:26.181283 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c269513-92f9-41f7-989a-669cb0822e0e None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:26.185663 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 437/857] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 10:20:26 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:26.186612 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c269513-92f9-41f7-989a-669cb0822e0e None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:26.207217 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f12440-3eb2-4816-b39e-bfb643b4bb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:26.208183 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f12440-3eb2-4816-b39e-bfb643b4bb31 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:26.208642 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f12440-3eb2-4816-b39e-bfb643b4bb31 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:26.209084 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f12440-3eb2-4816-b39e-bfb643b4bb31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:26.210150 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f12440-3eb2-4816-b39e-bfb643b4bb31 None admin] RBAC: Adding query filter params (role.id=532b37d9076c45618cf352aafb85f35e, scope.project.id=cff3a6011f4c4893bddffdad3d69f3df, user.id=5e6be34fc93a4551a83e5b8380188a6b) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:26.210750 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f12440-3eb2-4816-b39e-bfb643b4bb31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:26.212504 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f12440-3eb2-4816-b39e-bfb643b4bb31 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:26.226521 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 421/858] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:20:26 2022] GET /identity/v3/role_assignments?user.id=5e6be34fc93a4551a83e5b8380188a6b&scope.project.id=cff3a6011f4c4893bddffdad3d69f3df&role.id=532b37d9076c45618cf352aafb85f35e => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:26.317167 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c269513-92f9-41f7-989a-669cb0822e0e None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:26.359926 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2af8c-6748-4313-aff1-4cba41bb4178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:26.360588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2af8c-6748-4313-aff1-4cba41bb4178 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:26.360875 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2af8c-6748-4313-aff1-4cba41bb4178 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:26.361147 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2af8c-6748-4313-aff1-4cba41bb4178 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:26.374368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b2af8c-6748-4313-aff1-4cba41bb4178 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:20:26.380179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-22b2af8c-6748-4313-aff1-4cba41bb4178 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:20:26.380179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:26.380179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:26.380179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:26.380179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:26.380179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:26.380179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:26.380179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:26.380179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:26.380179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:26.380179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:26.380179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:26.380179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:20:26.380179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:20:26.380179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:20:26.380179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:20:26.381350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:20:26.381350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:20:26.381350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:20:26.381350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:20:26.381350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:26.381350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:26.381350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:26.381350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:26.381350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:26.381350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:26.381350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:26.381350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:26.381350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:26.381350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:26.381350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:26.381350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:26.382234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:26.382234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:26.382234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:26.382234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:26.382234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:20:26.382234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:20:26.382234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:20:26.382234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:20:26.382234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:20:26.382234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:20:26.382234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:20:26.382234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:26.382234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 438/859] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 10:20:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:26.388693 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f12440-3eb2-4816-b39e-bfb643b4bb31 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:26.464838 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:26.508467 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f12440-3eb2-4816-b39e-bfb643b4bb31 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:26.544599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65fe070-61da-4ce8-9bf2-740c6cb1eed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:26.545269 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65fe070-61da-4ce8-9bf2-740c6cb1eed8 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:26.545553 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65fe070-61da-4ce8-9bf2-740c6cb1eed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:26.545883 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65fe070-61da-4ce8-9bf2-740c6cb1eed8 None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:26.546585 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65fe070-61da-4ce8-9bf2-740c6cb1eed8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:26.546940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65fe070-61da-4ce8-9bf2-740c6cb1eed8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:26.548311 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65fe070-61da-4ce8-9bf2-740c6cb1eed8 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:26.560517 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 422/860] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 10:20:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:26.566609 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2af8c-6748-4313-aff1-4cba41bb4178 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:26.628649 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:26.677677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2af8c-6748-4313-aff1-4cba41bb4178 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:26.713175 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a349f730-5ed2-4866-b75e-6f7c9e4ed9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:26.713809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349f730-5ed2-4866-b75e-6f7c9e4ed9e2 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:26.714118 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349f730-5ed2-4866-b75e-6f7c9e4ed9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:26.714395 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349f730-5ed2-4866-b75e-6f7c9e4ed9e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:26.726052 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a349f730-5ed2-4866-b75e-6f7c9e4ed9e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:26.727459 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a349f730-5ed2-4866-b75e-6f7c9e4ed9e2 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:26.742782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-a349f730-5ed2-4866-b75e-6f7c9e4ed9e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:26.742782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:26.742782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:26.742782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:26.742782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:26.742782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:26.742782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:26.742782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:26.742782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:26.742782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:26.742782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:26.742782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:26.742782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:26.742782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:26.742782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:26.742782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:26.743743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:26.743743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:26.743743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:26.743743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:26.743743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:26.743743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:26.743743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:26.743743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:26.743743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:26.743743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:26.743743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:26.743743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:26.743743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:26.743743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:26.743743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:26.743743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:26.743743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:26.744629 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:26.744629 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:26.744629 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:26.744629 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 439/861] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:20:26.750261 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65fe070-61da-4ce8-9bf2-740c6cb1eed8 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:26.868900 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65fe070-61da-4ce8-9bf2-740c6cb1eed8 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:26.893086 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:26.906476 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b687a-8c37-4da3-8f6a-680338fc0b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:26.907356 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b687a-8c37-4da3-8f6a-680338fc0b3f None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:26.907925 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b687a-8c37-4da3-8f6a-680338fc0b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:26.908426 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b687a-8c37-4da3-8f6a-680338fc0b3f None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:26.909258 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b687a-8c37-4da3-8f6a-680338fc0b3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:26.909750 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b687a-8c37-4da3-8f6a-680338fc0b3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:26.911372 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b687a-8c37-4da3-8f6a-680338fc0b3f None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:26.925118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 423/862] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:26.930504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a349f730-5ed2-4866-b75e-6f7c9e4ed9e2 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:27.035702 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a349f730-5ed2-4866-b75e-6f7c9e4ed9e2 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:27.038166 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:27.072503 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7521af-cb01-4c32-9370-e164f28e9888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:27.073136 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7521af-cb01-4c32-9370-e164f28e9888 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:27.073412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7521af-cb01-4c32-9370-e164f28e9888 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:27.073684 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7521af-cb01-4c32-9370-e164f28e9888 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:27.082102 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7521af-cb01-4c32-9370-e164f28e9888 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:27.083546 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7521af-cb01-4c32-9370-e164f28e9888 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:27.096098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-ab7521af-cb01-4c32-9370-e164f28e9888 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 10:20:27.096098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:27.096098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:27.096098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:27.096098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:27.096098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:27.096098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:27.096098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:27.096098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:27.096098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:27.096098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:27.096098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:27.096098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:20:27.096098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:20:27.096098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:20:27.096098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:20:27.097074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:27.097074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:27.097074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:20:27.097074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:27.097074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:20:27.097074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:27.097074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:27.097074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:27.097074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:27.097074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:27.097074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:27.097074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:27.097074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:27.097074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:27.097074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:27.097074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:27.097074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:27.097074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:27.098012 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:27.098012 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:27.098012 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:20:27.098012 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:20:27.098012 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:20:27.098012 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:20:27.098012 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:20:27.098012 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:20:27.098012 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 10:20:27.098012 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:27.098012 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 440/863] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Sep 26 10:20:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:27.103210 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b687a-8c37-4da3-8f6a-680338fc0b3f None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:27.213572 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b687a-8c37-4da3-8f6a-680338fc0b3f None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:27.253340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee2be8-32d3-4cb6-92b3-50b6cc85987c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:27.254382 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee2be8-32d3-4cb6-92b3-50b6cc85987c None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:27.254870 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee2be8-32d3-4cb6-92b3-50b6cc85987c None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:27.255340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee2be8-32d3-4cb6-92b3-50b6cc85987c None admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:27.256257 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee2be8-32d3-4cb6-92b3-50b6cc85987c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:27.256757 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee2be8-32d3-4cb6-92b3-50b6cc85987c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:27.258417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee2be8-32d3-4cb6-92b3-50b6cc85987c None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:27.278940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 424/864] 10.222.0.49 () {56 vars in 1149 bytes} [Mon Sep 26 10:20:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:27.284339 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7521af-cb01-4c32-9370-e164f28e9888 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:27.396275 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7521af-cb01-4c32-9370-e164f28e9888 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:27.431375 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c61c4-bf0f-4749-90e8-ffb8cfb0d285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:27.431989 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c61c4-bf0f-4749-90e8-ffb8cfb0d285 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:27.432268 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c61c4-bf0f-4749-90e8-ffb8cfb0d285 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:27.432542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c61c4-bf0f-4749-90e8-ffb8cfb0d285 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:27.464118 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:27.503092 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c61c4-bf0f-4749-90e8-ffb8cfb0d285 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:27.504551 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c61c4-bf0f-4749-90e8-ffb8cfb0d285 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:27.611398 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 441/865] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:20:27 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 10:20:27.617153 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee2be8-32d3-4cb6-92b3-50b6cc85987c None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:27.666894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:27.706982 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cbb0b6-ccf2-404c-a64c-506140af6743 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:27.707297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cbb0b6-ccf2-404c-a64c-506140af6743 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:27.707547 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cbb0b6-ccf2-404c-a64c-506140af6743 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:27.708887 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 442/866] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:20:27.713929 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ba8b8-f241-4bd0-8e54-89cb3ea03b1d None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:27.714252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ba8b8-f241-4bd0-8e54-89cb3ea03b1d None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:27.714534 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ba8b8-f241-4bd0-8e54-89cb3ea03b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:27.730147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee2be8-32d3-4cb6-92b3-50b6cc85987c None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:27.733877 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-f88ba8b8-f241-4bd0-8e54-89cb3ea03b1d None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:27.735123 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-f88ba8b8-f241-4bd0-8e54-89cb3ea03b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:27.773457 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52316b-8510-47f2-97dc-c61253183ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:27.774282 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52316b-8510-47f2-97dc-c61253183ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:27.774564 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52316b-8510-47f2-97dc-c61253183ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:27.775656 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52316b-8510-47f2-97dc-c61253183ad3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:27.788834 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52316b-8510-47f2-97dc-c61253183ad3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:27.791114 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52316b-8510-47f2-97dc-c61253183ad3 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:27.809091 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-7a52316b-8510-47f2-97dc-c61253183ad3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:27.809091 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:27.809091 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:27.809091 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:27.809091 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:27.809091 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:27.809091 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:27.809091 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:27.809091 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:27.809091 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:27.809091 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:27.809091 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:27.809091 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:27.809091 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:27.809091 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:27.809091 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:27.810367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:27.810367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:27.810367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:27.810367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:27.810367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:27.810367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:27.810367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:27.810367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:27.810367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:27.810367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:27.810367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:27.810367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:27.810367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:27.810367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:27.810367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:27.810367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:27.810367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:27.811306 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:27.811306 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:27.811306 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:27.811306 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 425/867] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:27.815987 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52316b-8510-47f2-97dc-c61253183ad3 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:27.816854 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:27.828116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 443/868] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:27 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 10:20:27.923647 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:27.931556 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52316b-8510-47f2-97dc-c61253183ad3 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:27.967647 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4dcadc-6b0c-4cc5-9406-d55b4ed87d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:27.968256 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4dcadc-6b0c-4cc5-9406-d55b4ed87d8a None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:27.968536 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4dcadc-6b0c-4cc5-9406-d55b4ed87d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:27.968812 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4dcadc-6b0c-4cc5-9406-d55b4ed87d8a None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:27.969454 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4dcadc-6b0c-4cc5-9406-d55b4ed87d8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:27.969755 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4dcadc-6b0c-4cc5-9406-d55b4ed87d8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:27.971213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4dcadc-6b0c-4cc5-9406-d55b4ed87d8a None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:27.984302 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 426/869] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:27.989751 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ba8b8-f241-4bd0-8e54-89cb3ea03b1d None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:28.098463 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ba8b8-f241-4bd0-8e54-89cb3ea03b1d None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:28.116367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4dcadc-6b0c-4cc5-9406-d55b4ed87d8a None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:28.137295 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8710466-7557-4a98-964e-cc6747d546c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:28.138008 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8710466-7557-4a98-964e-cc6747d546c9 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:28.138319 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8710466-7557-4a98-964e-cc6747d546c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:28.138591 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8710466-7557-4a98-964e-cc6747d546c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:28.147275 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8710466-7557-4a98-964e-cc6747d546c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:28.148988 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8710466-7557-4a98-964e-cc6747d546c9 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:28.163101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-e8710466-7557-4a98-964e-cc6747d546c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:28.163101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:28.163101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:28.163101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:28.163101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:28.163101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:28.163101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:28.163101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:28.163101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:28.163101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:28.163101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:28.163101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:28.163101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:20:28.163101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:20:28.163101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:20:28.164047 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:20:28.164047 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:28.164047 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:28.164047 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:28.164047 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:28.164047 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:28.164047 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:28.164047 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:28.164047 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:28.164047 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:28.164047 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:28.164047 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:28.164047 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:28.164047 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:28.164047 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:28.164047 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:28.164047 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:28.164972 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:20:28.164972 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:20:28.164972 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:20:28.164972 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:20:28.164972 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:20:28.164972 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:20:28.164972 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:28.164972 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:28.164972 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 444/870] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:20:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:28.170723 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8710466-7557-4a98-964e-cc6747d546c9 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:28.230700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4dcadc-6b0c-4cc5-9406-d55b4ed87d8a None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:28.246059 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:28.271742 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff2f0f-7a0c-4653-ac8f-a63101b2084c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:28.272946 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff2f0f-7a0c-4653-ac8f-a63101b2084c None admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:28.273448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff2f0f-7a0c-4653-ac8f-a63101b2084c None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:28.273922 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff2f0f-7a0c-4653-ac8f-a63101b2084c None admin] PATH_INFO: `/v3/services` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:28.274811 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ff2f0f-7a0c-4653-ac8f-a63101b2084c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:28.276589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ff2f0f-7a0c-4653-ac8f-a63101b2084c None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:28.301615 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8710466-7557-4a98-964e-cc6747d546c9 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:28.302012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 427/871] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Sep 26 10:20:28 2022] POST /identity/v3/services => generated 269 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:20:28.345634 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb643c-a39d-4281-8c8c-4e66bf8159b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:28.346294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb643c-a39d-4281-8c8c-4e66bf8159b6 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:28.346576 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb643c-a39d-4281-8c8c-4e66bf8159b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:28.346846 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb643c-a39d-4281-8c8c-4e66bf8159b6 None admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:28.347544 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deb643c-a39d-4281-8c8c-4e66bf8159b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:28.347847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deb643c-a39d-4281-8c8c-4e66bf8159b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:28.349281 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deb643c-a39d-4281-8c8c-4e66bf8159b6 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:28.364575 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 445/872] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:20:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:28.370677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff2f0f-7a0c-4653-ac8f-a63101b2084c None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:28.388672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:28.494623 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff2f0f-7a0c-4653-ac8f-a63101b2084c None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:28.537591 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355900d3-3258-4945-9ae8-001f2895ddd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:28.538509 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355900d3-3258-4945-9ae8-001f2895ddd2 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:28.538942 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355900d3-3258-4945-9ae8-001f2895ddd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:28.539462 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355900d3-3258-4945-9ae8-001f2895ddd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:28.540339 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355900d3-3258-4945-9ae8-001f2895ddd2 None admin] RBAC: Adding query filter params (role.id=d36e58344b604bcda1cee49065e8497e, scope.project.id=cff3a6011f4c4893bddffdad3d69f3df, user.id=9158557f1a2947e2815dc8d586d8b0c3) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:28.540805 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355900d3-3258-4945-9ae8-001f2895ddd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:28.542271 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355900d3-3258-4945-9ae8-001f2895ddd2 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:28.548953 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 428/873] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:20:28 2022] GET /identity/v3/role_assignments?user.id=9158557f1a2947e2815dc8d586d8b0c3&scope.project.id=cff3a6011f4c4893bddffdad3d69f3df&role.id=d36e58344b604bcda1cee49065e8497e => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:28.594263 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532f478-df56-48ce-8773-9d4ed1912499 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:28.594610 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532f478-df56-48ce-8773-9d4ed1912499 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:28.594821 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532f478-df56-48ce-8773-9d4ed1912499 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:28.596667 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 446/874] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:28.602761 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ba957-a9bb-40a8-8047-5bfbcfef316c None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:28.603089 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ba957-a9bb-40a8-8047-5bfbcfef316c None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:28.603315 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ba957-a9bb-40a8-8047-5bfbcfef316c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:28.624739 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-678ba957-a9bb-40a8-8047-5bfbcfef316c None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:28.625971 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-678ba957-a9bb-40a8-8047-5bfbcfef316c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:28.704449 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 429/875] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:28 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:20:28.713816 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ab48d-7e1d-490b-83f3-a6b5c9738219 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:28.714140 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ab48d-7e1d-490b-83f3-a6b5c9738219 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:28.714393 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ab48d-7e1d-490b-83f3-a6b5c9738219 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:28.716565 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 447/876] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:28.722306 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b4a40-9750-4719-b6f1-19da26c443d5 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:28.722612 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b4a40-9750-4719-b6f1-19da26c443d5 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:28.722850 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b4a40-9750-4719-b6f1-19da26c443d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:28.742305 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-cc7b4a40-9750-4719-b6f1-19da26c443d5 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:28.743542 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-cc7b4a40-9750-4719-b6f1-19da26c443d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:28.745954 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:28.827259 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 430/877] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:28 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:20:28.982139 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ab48d-7e1d-490b-83f3-a6b5c9738219 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:29.094708 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ab48d-7e1d-490b-83f3-a6b5c9738219 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:29.111390 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:29.134712 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b4a40-9750-4719-b6f1-19da26c443d5 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:29.138655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290ae13-0752-4c65-a27e-323c4d4c2da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:29.139283 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290ae13-0752-4c65-a27e-323c4d4c2da2 None admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:29.139579 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290ae13-0752-4c65-a27e-323c4d4c2da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:29.139827 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290ae13-0752-4c65-a27e-323c4d4c2da2 None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:29.141677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f290ae13-0752-4c65-a27e-323c4d4c2da2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:29.143128 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f290ae13-0752-4c65-a27e-323c4d4c2da2 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:29.170071 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 448/878] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Sep 26 10:20:28 2022] POST /identity/v3/services => generated 251 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:20:29.268571 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b4a40-9750-4719-b6f1-19da26c443d5 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:29.321229 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dde772-898a-41d0-8b88-09a72c199098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:29.321939 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dde772-898a-41d0-8b88-09a72c199098 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:29.322309 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dde772-898a-41d0-8b88-09a72c199098 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:29.322581 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dde772-898a-41d0-8b88-09a72c199098 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:29.323189 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dde772-898a-41d0-8b88-09a72c199098 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:29.324726 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dde772-898a-41d0-8b88-09a72c199098 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:29.327045 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:29.337649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-93dde772-898a-41d0-8b88-09a72c199098 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 10:20:29.337649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:29.337649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 10:20:29.337649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 10:20:29.337649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 10:20:29.337649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 10:20:29.337649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 10:20:29.337649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:29.337649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 10:20:29.337649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:29.337649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 10:20:29.337649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:29.337649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:29.337649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:29.337649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:29.337649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:29.337649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:29.338765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:29.338765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:29.338765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:29.338765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:29.338765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:29.338765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:29.338765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 10:20:29.338765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 10:20:29.338765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 10:20:29.338765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 10:20:29.338765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:29.338765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:29.338765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:29.338765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:29.338765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:29.338765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:29.339846 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:29.339846 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:29.339846 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:29.339846 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:29.339846 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:29.339846 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:29.339846 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:29.339846 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:29.339846 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:29.339846 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:29.339846 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 10:20:29.339846 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:29.339846 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 10:20:29.339846 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:29.339846 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 431/879] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Sep 26 10:20:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:29.351253 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290ae13-0752-4c65-a27e-323c4d4c2da2 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:29.463886 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290ae13-0752-4c65-a27e-323c4d4c2da2 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:29.501556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65b92e-b622-424e-8d7a-02f6a04ff3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:29.502204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65b92e-b622-424e-8d7a-02f6a04ff3c7 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:29.502486 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65b92e-b622-424e-8d7a-02f6a04ff3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:29.502762 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65b92e-b622-424e-8d7a-02f6a04ff3c7 None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:29.503393 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea65b92e-b622-424e-8d7a-02f6a04ff3c7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:29.503696 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea65b92e-b622-424e-8d7a-02f6a04ff3c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:29.505054 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea65b92e-b622-424e-8d7a-02f6a04ff3c7 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:29.510218 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 449/880] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Sep 26 10:20:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:29.516133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dde772-898a-41d0-8b88-09a72c199098 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:29.621910 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dde772-898a-41d0-8b88-09a72c199098 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:29.660069 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c3b80-764a-4b97-85c1-73915df4b5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:29.661777 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c3b80-764a-4b97-85c1-73915df4b5b1 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:29.662309 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c3b80-764a-4b97-85c1-73915df4b5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:29.662810 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c3b80-764a-4b97-85c1-73915df4b5b1 None admin] PATH_INFO: `/v3/services` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:29.663694 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c3b80-764a-4b97-85c1-73915df4b5b1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:29.664195 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c3b80-764a-4b97-85c1-73915df4b5b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:29.665789 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c3b80-764a-4b97-85c1-73915df4b5b1 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:29.678167 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 432/881] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Sep 26 10:20:29 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:30.866513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11d1e9-1b8c-44db-8c0d-b356b05b7c2a None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:30.867037 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11d1e9-1b8c-44db-8c0d-b356b05b7c2a None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:30.867037 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11d1e9-1b8c-44db-8c0d-b356b05b7c2a None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:30.868645 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 450/882] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:30.874403 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3bc92-61e1-4297-a206-876805b94b30 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:30.874868 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3bc92-61e1-4297-a206-876805b94b30 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:30.874957 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3bc92-61e1-4297-a206-876805b94b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:30.893701 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-bbd3bc92-61e1-4297-a206-876805b94b30 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:30.894915 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-bbd3bc92-61e1-4297-a206-876805b94b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:30.960159 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df805f7-1602-435d-8ece-00a7102d2446 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:30.960515 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df805f7-1602-435d-8ece-00a7102d2446 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:30.960724 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df805f7-1602-435d-8ece-00a7102d2446 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:30.961034 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:30.962441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 451/883] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:30.968090 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57720b-408a-453f-9d89-5ef7d32355a0 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:30.968379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57720b-408a-453f-9d89-5ef7d32355a0 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:30.968660 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57720b-408a-453f-9d89-5ef7d32355a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:30.975870 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 433/884] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:30 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:20:30.987756 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-7b57720b-408a-453f-9d89-5ef7d32355a0 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:30.988968 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-7b57720b-408a-453f-9d89-5ef7d32355a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:31.023858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:31.071228 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 452/885] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:30 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 10:20:31.252842 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3bc92-61e1-4297-a206-876805b94b30 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:31.341134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b57720b-408a-453f-9d89-5ef7d32355a0 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:31.362661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3bc92-61e1-4297-a206-876805b94b30 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:31.412539 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c23f04-c52e-4249-a3a1-820d8b631fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:31.414173 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c23f04-c52e-4249-a3a1-820d8b631fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:31.414808 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c23f04-c52e-4249-a3a1-820d8b631fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:31.415442 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c23f04-c52e-4249-a3a1-820d8b631fb2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:31.416280 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c23f04-c52e-4249-a3a1-820d8b631fb2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:31.418279 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c23f04-c52e-4249-a3a1-820d8b631fb2 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:31.431799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-c0c23f04-c52e-4249-a3a1-820d8b631fb2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 10:20:31.431799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:31.431799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 10:20:31.431799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 10:20:31.431799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 10:20:31.431799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 10:20:31.431799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 10:20:31.431799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:31.431799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 10:20:31.431799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:31.431799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 10:20:31.431799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:31.431799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:31.431799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:31.431799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:31.431799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:31.432824 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:31.432824 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:31.432824 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:31.432824 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:31.432824 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:31.432824 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:31.432824 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:31.432824 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 10:20:31.432824 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 10:20:31.432824 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 10:20:31.432824 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 10:20:31.432824 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:31.432824 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:31.432824 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:31.432824 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:31.432824 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:31.432824 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:31.433742 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:31.433742 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:31.433742 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:31.433742 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:31.433742 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:31.433742 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:31.433742 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:31.433742 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:31.433742 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:31.433742 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:31.433742 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 10:20:31.433742 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:31.433742 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 10:20:31.433742 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:31.440666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 434/886] 10.222.0.49 () {56 vars in 1132 bytes} [Mon Sep 26 10:20:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:31.442373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c23f04-c52e-4249-a3a1-820d8b631fb2 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:31.460806 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b57720b-408a-453f-9d89-5ef7d32355a0 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:31.501267 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07b673-43dd-4499-b103-4ba349e40f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:31.502294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07b673-43dd-4499-b103-4ba349e40f00 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:31.502777 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07b673-43dd-4499-b103-4ba349e40f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:31.503288 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07b673-43dd-4499-b103-4ba349e40f00 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:31.519092 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07b673-43dd-4499-b103-4ba349e40f00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:20:31.522375 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:31.525456 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-7e07b673-43dd-4499-b103-4ba349e40f00 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 10:20:31.525456 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:31.525456 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:31.525456 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:31.525456 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:31.525456 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:31.525456 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:31.525456 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:31.525456 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:31.525456 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:31.525456 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:31.525456 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:31.525456 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:20:31.525456 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:20:31.525456 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:20:31.525456 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:20:31.526742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:20:31.526742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:20:31.526742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:20:31.526742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:20:31.526742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:31.526742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:31.526742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:31.526742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:31.526742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:31.526742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:31.526742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:31.526742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:31.526742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:31.526742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:31.526742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:31.526742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:31.527568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:31.527568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:31.527568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:31.527568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:31.527568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:20:31.527568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:20:31.527568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:20:31.527568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:20:31.527568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:20:31.527568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:20:31.527568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 10:20:31.527568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:31.527568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 453/887] 10.222.0.49 () {56 vars in 1120 bytes} [Mon Sep 26 10:20:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:31.528258 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb31f36-996f-4578-9ed8-265ac22bde39 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:31.528258 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb31f36-996f-4578-9ed8-265ac22bde39 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:31.529065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb31f36-996f-4578-9ed8-265ac22bde39 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:31.530661 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 454/888] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:31.534997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb31f36-996f-4578-9ed8-265ac22bde39 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:31.562522 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:31.576938 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c23f04-c52e-4249-a3a1-820d8b631fb2 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:31.612843 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f9eca-6657-4c1d-ba9a-b69623497502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:31.613712 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f9eca-6657-4c1d-ba9a-b69623497502 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:31.613989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f9eca-6657-4c1d-ba9a-b69623497502 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:31.614288 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f9eca-6657-4c1d-ba9a-b69623497502 None admin] PATH_INFO: `/v3/services` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:31.614907 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509f9eca-6657-4c1d-ba9a-b69623497502 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:31.615204 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509f9eca-6657-4c1d-ba9a-b69623497502 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:31.616554 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509f9eca-6657-4c1d-ba9a-b69623497502 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:31.621372 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 435/889] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Sep 26 10:20:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:31.622452 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a82f89-c458-4f05-8146-876722304194 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:31.622727 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a82f89-c458-4f05-8146-876722304194 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:31.622994 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a82f89-c458-4f05-8146-876722304194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:31.642477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-26a82f89-c458-4f05-8146-876722304194 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:31.643663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-26a82f89-c458-4f05-8146-876722304194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:31.645728 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb31f36-996f-4578-9ed8-265ac22bde39 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:31.679982 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0341a3-1622-4171-add2-a3a6f9280ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:31.680592 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0341a3-1622-4171-add2-a3a6f9280ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:31.680865 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0341a3-1622-4171-add2-a3a6f9280ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:31.681134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0341a3-1622-4171-add2-a3a6f9280ad1 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:31.681792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0341a3-1622-4171-add2-a3a6f9280ad1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:31.682120 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0341a3-1622-4171-add2-a3a6f9280ad1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:31.683490 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0341a3-1622-4171-add2-a3a6f9280ad1 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:31.691660 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 455/890] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Sep 26 10:20:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:31.694913 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0341a3-1622-4171-add2-a3a6f9280ad1 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:31.715258 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 436/891] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:31 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:20:31.716657 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a82f89-c458-4f05-8146-876722304194 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:31.786914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:31.797020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:31.819114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0341a3-1622-4171-add2-a3a6f9280ad1 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:31.859671 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a82f89-c458-4f05-8146-876722304194 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:31.867666 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d010aa9-cdf7-4b41-b059-3c232f9323f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:31.868343 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d010aa9-cdf7-4b41-b059-3c232f9323f1 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:31.868710 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d010aa9-cdf7-4b41-b059-3c232f9323f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:31.869076 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d010aa9-cdf7-4b41-b059-3c232f9323f1 None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:31.870350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d010aa9-cdf7-4b41-b059-3c232f9323f1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:31.870749 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d010aa9-cdf7-4b41-b059-3c232f9323f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:31.873183 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d010aa9-cdf7-4b41-b059-3c232f9323f1 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:31.880467 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 456/892] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Sep 26 10:20:31 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:31.887463 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d010aa9-cdf7-4b41-b059-3c232f9323f1 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:31.901205 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef00d067-4f01-4544-abca-d435be51c885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:31.901846 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00d067-4f01-4544-abca-d435be51c885 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:31.902167 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00d067-4f01-4544-abca-d435be51c885 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:31.902447 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00d067-4f01-4544-abca-d435be51c885 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:31.913150 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef00d067-4f01-4544-abca-d435be51c885 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:31.914696 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef00d067-4f01-4544-abca-d435be51c885 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:31.929923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-ef00d067-4f01-4544-abca-d435be51c885 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:31.929923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:31.929923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:31.929923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:31.929923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:31.929923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:31.929923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:31.929923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:31.929923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:31.929923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:31.929923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:31.929923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:31.929923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:31.929923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:31.929923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:31.929923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:31.930913 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:31.930913 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:31.930913 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:31.930913 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:31.930913 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:31.930913 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:31.930913 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:31.930913 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:31.930913 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:31.930913 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:31.930913 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:31.930913 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:31.930913 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:31.930913 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:31.930913 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:31.930913 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:31.930913 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:31.931821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:31.931821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:31.931821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:31.931821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 437/893] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:31.932420 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc0eaf-c3f5-4f5d-99ea-4961b1895c34 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:31.932688 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc0eaf-c3f5-4f5d-99ea-4961b1895c34 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:31.932955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc0eaf-c3f5-4f5d-99ea-4961b1895c34 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:31.934455 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 438/894] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:31.936920 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbc0eaf-c3f5-4f5d-99ea-4961b1895c34 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:32.005216 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d010aa9-cdf7-4b41-b059-3c232f9323f1 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:32.043418 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0995e5-3747-49ad-8b40-b2038fb2494a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:32.044043 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0995e5-3747-49ad-8b40-b2038fb2494a None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:32.044319 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0995e5-3747-49ad-8b40-b2038fb2494a None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:32.044595 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0995e5-3747-49ad-8b40-b2038fb2494a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:32.045284 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0995e5-3747-49ad-8b40-b2038fb2494a None admin] RBAC: Adding query filter params (interface=public, service_id=2cd99cec73aa41e981ce7d9973ea6ad2, region_id=RegionOne) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:32.045586 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0995e5-3747-49ad-8b40-b2038fb2494a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:32.046974 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0995e5-3747-49ad-8b40-b2038fb2494a None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:32.051107 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbc0eaf-c3f5-4f5d-99ea-4961b1895c34 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:32.052642 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:32.054692 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 457/895] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Sep 26 10:20:31 2022] GET /identity/v3/endpoints?service_id=2cd99cec73aa41e981ce7d9973ea6ad2&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:32.055836 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0235d09-cdaa-4a47-a17b-439c69d22b19 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:32.056122 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0235d09-cdaa-4a47-a17b-439c69d22b19 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:32.056385 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0235d09-cdaa-4a47-a17b-439c69d22b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:32.074884 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-f0235d09-cdaa-4a47-a17b-439c69d22b19 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:32.076660 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-f0235d09-cdaa-4a47-a17b-439c69d22b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:32.085972 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:32.087333 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4debe-ada8-4c57-9dcc-65857f74d115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:32.087914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4debe-ada8-4c57-9dcc-65857f74d115 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:32.088223 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4debe-ada8-4c57-9dcc-65857f74d115 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:32.088524 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4debe-ada8-4c57-9dcc-65857f74d115 None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:32.089082 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e4debe-ada8-4c57-9dcc-65857f74d115 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:32.089383 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e4debe-ada8-4c57-9dcc-65857f74d115 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:32.090773 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e4debe-ada8-4c57-9dcc-65857f74d115 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:32.102567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 439/896] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:32.103384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4debe-ada8-4c57-9dcc-65857f74d115 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:32.148319 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 458/897] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:32 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:20:32.148997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0235d09-cdaa-4a47-a17b-439c69d22b19 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:32.205122 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4debe-ada8-4c57-9dcc-65857f74d115 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:32.249065 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c77a34-c1b4-4023-a9e7-cedf8930fec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:32.249777 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c77a34-c1b4-4023-a9e7-cedf8930fec3 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:32.249997 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c77a34-c1b4-4023-a9e7-cedf8930fec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:32.250295 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c77a34-c1b4-4023-a9e7-cedf8930fec3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:32.250871 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c77a34-c1b4-4023-a9e7-cedf8930fec3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:32.252215 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c77a34-c1b4-4023-a9e7-cedf8930fec3 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:32.255444 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0235d09-cdaa-4a47-a17b-439c69d22b19 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:32.261184 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-00c77a34-c1b4-4023-a9e7-cedf8930fec3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 10:20:32.261184 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:32.261184 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 10:20:32.261184 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 10:20:32.261184 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 10:20:32.261184 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 10:20:32.261184 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 10:20:32.261184 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:32.261184 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 10:20:32.261184 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:32.261184 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 10:20:32.261184 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:32.261184 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:32.261184 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:32.261184 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:32.261184 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:32.261184 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:32.262203 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:32.262203 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:32.262203 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:32.262203 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:32.262203 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:32.262203 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:32.262203 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 10:20:32.262203 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 10:20:32.262203 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 10:20:32.262203 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 10:20:32.262203 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:32.262203 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:32.262203 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:32.262203 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:32.262203 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:32.262203 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:32.267087 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:32.267087 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:32.267087 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:32.267087 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:32.267087 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:32.267087 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:32.267087 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:32.267087 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:32.267087 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:32.267087 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:32.267087 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 10:20:32.267087 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:32.267087 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 10:20:32.267087 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:32.267087 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 440/898] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 10:20:32 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:32.267889 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c77a34-c1b4-4023-a9e7-cedf8930fec3 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:32.294570 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950592f8-9b43-4950-ac74-441da77396b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:32.295175 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950592f8-9b43-4950-ac74-441da77396b8 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:32.295459 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950592f8-9b43-4950-ac74-441da77396b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:32.295720 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950592f8-9b43-4950-ac74-441da77396b8 None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:32.296322 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950592f8-9b43-4950-ac74-441da77396b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:32.297663 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950592f8-9b43-4950-ac74-441da77396b8 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:32.299026 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:32.303964 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 459/899] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Sep 26 10:20:32 2022] GET /identity/v3/services => generated 1268 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:20:32.303964 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950592f8-9b43-4950-ac74-441da77396b8 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:32.322659 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:32.423742 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c77a34-c1b4-4023-a9e7-cedf8930fec3 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:32.430969 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950592f8-9b43-4950-ac74-441da77396b8 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:32.462439 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c156ab-07a6-4af4-b310-d11698f15fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:32.463121 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c156ab-07a6-4af4-b310-d11698f15fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:32.463400 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c156ab-07a6-4af4-b310-d11698f15fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:32.463670 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c156ab-07a6-4af4-b310-d11698f15fe4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:32.472094 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c156ab-07a6-4af4-b310-d11698f15fe4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:32.473559 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c156ab-07a6-4af4-b310-d11698f15fe4 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:32.474469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d6a22-32f7-494a-bb56-5660d4510bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:32.475086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d6a22-32f7-494a-bb56-5660d4510bdd None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:32.475469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d6a22-32f7-494a-bb56-5660d4510bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:32.475661 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d6a22-32f7-494a-bb56-5660d4510bdd None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:32.476300 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d6a22-32f7-494a-bb56-5660d4510bdd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:32.477155 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d6a22-32f7-494a-bb56-5660d4510bdd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:32.478061 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d6a22-32f7-494a-bb56-5660d4510bdd None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:32.488334 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 460/900] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 10:20:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:32.488334 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d6a22-32f7-494a-bb56-5660d4510bdd None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:32.488630 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-07c156ab-07a6-4af4-b310-d11698f15fe4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 10:20:32.488630 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:32.488630 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:32.488630 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:32.488630 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:32.488630 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:32.488630 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:32.488630 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:32.488630 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:32.488630 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:32.488630 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:32.488630 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:32.488630 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:20:32.488630 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:20:32.488630 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:20:32.488630 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:20:32.490669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:32.490669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:32.490669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:20:32.490669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:32.490669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:20:32.490669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:32.490669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:32.490669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:32.490669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:32.490669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:32.490669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:32.490669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:32.490669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:32.490669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:32.490669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:32.490669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:32.490669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:32.490669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:32.492074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:32.492074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:32.492074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:20:32.492074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:20:32.492074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:20:32.492074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:20:32.492074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:20:32.492074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:20:32.492074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 10:20:32.492074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:32.492074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 441/901] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Sep 26 10:20:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:32.492074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c156ab-07a6-4af4-b310-d11698f15fe4 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:32.572436 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:32.589726 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:32.605640 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d6a22-32f7-494a-bb56-5660d4510bdd None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:32.606759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c156ab-07a6-4af4-b310-d11698f15fe4 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:32.641513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074df217-ba8b-4b22-b945-cbf72465adf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:32.642139 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074df217-ba8b-4b22-b945-cbf72465adf3 None admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:32.642415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074df217-ba8b-4b22-b945-cbf72465adf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:32.642522 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f38613-1739-4fe1-81e3-72fb38d68ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:32.642701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074df217-ba8b-4b22-b945-cbf72465adf3 None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:32.643128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f38613-1739-4fe1-81e3-72fb38d68ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:32.643312 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074df217-ba8b-4b22-b945-cbf72465adf3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:32.643417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f38613-1739-4fe1-81e3-72fb38d68ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:32.643699 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f38613-1739-4fe1-81e3-72fb38d68ed5 None admin] PATH_INFO: `/v3/services` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:32.644328 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f38613-1739-4fe1-81e3-72fb38d68ed5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:32.644634 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f38613-1739-4fe1-81e3-72fb38d68ed5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:32.644731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074df217-ba8b-4b22-b945-cbf72465adf3 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:32.645982 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f38613-1739-4fe1-81e3-72fb38d68ed5 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:32.654184 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 442/902] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 10:20:32 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:32.655135 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f38613-1739-4fe1-81e3-72fb38d68ed5 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:32.664657 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 461/903] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Sep 26 10:20:32 2022] POST /identity/v3/services => generated 241 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:20:32.665778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074df217-ba8b-4b22-b945-cbf72465adf3 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:32.784781 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f38613-1739-4fe1-81e3-72fb38d68ed5 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:32.830199 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074df217-ba8b-4b22-b945-cbf72465adf3 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:32.834974 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac253c-a78c-4c5e-9f5d-c15b3f30ccd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:32.835187 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac253c-a78c-4c5e-9f5d-c15b3f30ccd2 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:32.835407 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac253c-a78c-4c5e-9f5d-c15b3f30ccd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:32.835663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac253c-a78c-4c5e-9f5d-c15b3f30ccd2 None admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:32.836403 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac253c-a78c-4c5e-9f5d-c15b3f30ccd2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:32.836708 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac253c-a78c-4c5e-9f5d-c15b3f30ccd2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:32.838193 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac253c-a78c-4c5e-9f5d-c15b3f30ccd2 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:32.861196 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 443/904] 10.222.0.49 () {56 vars in 1149 bytes} [Mon Sep 26 10:20:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:32.867642 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac253c-a78c-4c5e-9f5d-c15b3f30ccd2 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:32.868949 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:32.881613 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:32.882777 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de948c-2f05-46ab-bdfe-df26c00a34c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:32.883664 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de948c-2f05-46ab-bdfe-df26c00a34c9 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:32.883967 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de948c-2f05-46ab-bdfe-df26c00a34c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:32.884209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de948c-2f05-46ab-bdfe-df26c00a34c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:32.885129 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de948c-2f05-46ab-bdfe-df26c00a34c9 None admin] RBAC: Adding query filter params (interface=public, service_id=e152d6f6254447feb511d67d3a853b3d, region_id=RegionOne) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:32.885436 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de948c-2f05-46ab-bdfe-df26c00a34c9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:32.887806 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de948c-2f05-46ab-bdfe-df26c00a34c9 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:32.896033 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 462/905] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Sep 26 10:20:32 2022] GET /identity/v3/endpoints?service_id=e152d6f6254447feb511d67d3a853b3d&interface=public®ion_id=RegionOne => generated 194 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:32.901333 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de948c-2f05-46ab-bdfe-df26c00a34c9 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:33.000552 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac253c-a78c-4c5e-9f5d-c15b3f30ccd2 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:33.016116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de948c-2f05-46ab-bdfe-df26c00a34c9 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:33.049821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0188c8-dbec-44cb-8a9e-32fcea3b25ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:33.050732 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0188c8-dbec-44cb-8a9e-32fcea3b25ea None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:33.051125 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0188c8-dbec-44cb-8a9e-32fcea3b25ea None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:33.051459 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0188c8-dbec-44cb-8a9e-32fcea3b25ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:33.057824 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba6265-b4a2-46d8-95ac-34f34fd44398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:33.058727 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba6265-b4a2-46d8-95ac-34f34fd44398 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:33.059197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba6265-b4a2-46d8-95ac-34f34fd44398 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:33.059717 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba6265-b4a2-46d8-95ac-34f34fd44398 None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:33.061003 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ba6265-b4a2-46d8-95ac-34f34fd44398 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:33.062748 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ba6265-b4a2-46d8-95ac-34f34fd44398 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:33.069688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 463/906] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Sep 26 10:20:32 2022] GET /identity/v3/services => generated 1497 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:20:33.132589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0188c8-dbec-44cb-8a9e-32fcea3b25ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:33.134221 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0188c8-dbec-44cb-8a9e-32fcea3b25ea None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:33.145747 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:33.253592 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 444/907] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:20:32 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 10:20:33.259753 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba6265-b4a2-46d8-95ac-34f34fd44398 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:33.330995 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:33.380310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba6265-b4a2-46d8-95ac-34f34fd44398 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:33.416596 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac0c70b-3f21-4c8e-9d0a-e4f947c22acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:33.417224 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0c70b-3f21-4c8e-9d0a-e4f947c22acb None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:33.417499 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0c70b-3f21-4c8e-9d0a-e4f947c22acb None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:33.417767 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0c70b-3f21-4c8e-9d0a-e4f947c22acb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:33.429465 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac0c70b-3f21-4c8e-9d0a-e4f947c22acb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:33.430929 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac0c70b-3f21-4c8e-9d0a-e4f947c22acb None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:33.445558 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-eac0c70b-3f21-4c8e-9d0a-e4f947c22acb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:33.445558 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:33.445558 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:33.445558 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:33.445558 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:33.445558 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:33.445558 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:33.445558 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:33.445558 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:33.445558 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:33.445558 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:33.445558 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:33.445558 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:33.445558 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:33.445558 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:33.445558 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:33.446646 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:33.446646 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:33.446646 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:33.446646 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:33.446646 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:33.446646 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:33.446646 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:33.446646 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:33.446646 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:33.446646 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:33.446646 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:33.446646 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:33.446646 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:33.446646 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:33.446646 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:33.446646 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:33.446646 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:33.447623 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:33.447623 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:33.447623 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:33.447623 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 464/908] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:33.455901 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0188c8-dbec-44cb-8a9e-32fcea3b25ea None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:33.562659 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0188c8-dbec-44cb-8a9e-32fcea3b25ea None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:33.574514 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:33.600635 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26aad3-02b5-4867-bc84-dbf70ed54579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:33.601165 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26aad3-02b5-4867-bc84-dbf70ed54579 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:33.601445 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26aad3-02b5-4867-bc84-dbf70ed54579 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:33.601717 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26aad3-02b5-4867-bc84-dbf70ed54579 None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:33.602394 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d26aad3-02b5-4867-bc84-dbf70ed54579 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:33.602695 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d26aad3-02b5-4867-bc84-dbf70ed54579 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:33.604053 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d26aad3-02b5-4867-bc84-dbf70ed54579 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:33.616840 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 445/909] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:33.623215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac0c70b-3f21-4c8e-9d0a-e4f947c22acb None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:33.733612 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac0c70b-3f21-4c8e-9d0a-e4f947c22acb None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:33.743070 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:33.768962 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e28096-4415-4a8d-b5bb-2595b6fcd714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:33.769593 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e28096-4415-4a8d-b5bb-2595b6fcd714 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:33.770160 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e28096-4415-4a8d-b5bb-2595b6fcd714 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:33.770863 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e28096-4415-4a8d-b5bb-2595b6fcd714 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:33.778212 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e28096-4415-4a8d-b5bb-2595b6fcd714 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:33.780198 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e28096-4415-4a8d-b5bb-2595b6fcd714 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:33.793020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-f5e28096-4415-4a8d-b5bb-2595b6fcd714 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:33.793020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:33.793020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:33.793020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:33.793020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:33.793020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:33.793020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:33.793020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:33.793020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:33.793020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:33.793020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:33.793020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:33.793020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:20:33.793020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:20:33.793020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:20:33.793925 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:20:33.793925 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:33.793925 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:33.793925 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:33.793925 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:33.793925 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:33.793925 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:33.793925 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:33.793925 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:33.793925 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:33.793925 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:33.793925 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:33.793925 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:33.793925 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:33.793925 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:33.793925 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:33.793925 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:33.794970 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:20:33.794970 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:20:33.794970 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:20:33.794970 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:20:33.794970 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:20:33.794970 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:20:33.794970 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:33.794970 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:33.794970 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 465/910] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:20:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:33.800700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26aad3-02b5-4867-bc84-dbf70ed54579 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:33.907654 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26aad3-02b5-4867-bc84-dbf70ed54579 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:33.944097 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcda4fe-c672-4c9c-9efa-4f8c9b3fc977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:33.944977 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcda4fe-c672-4c9c-9efa-4f8c9b3fc977 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:33.945401 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcda4fe-c672-4c9c-9efa-4f8c9b3fc977 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:33.945830 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcda4fe-c672-4c9c-9efa-4f8c9b3fc977 None admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:33.946700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcda4fe-c672-4c9c-9efa-4f8c9b3fc977 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:33.947164 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcda4fe-c672-4c9c-9efa-4f8c9b3fc977 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:33.948732 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcda4fe-c672-4c9c-9efa-4f8c9b3fc977 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:33.962061 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 446/911] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:20:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:33.968230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e28096-4415-4a8d-b5bb-2595b6fcd714 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:34.079526 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e28096-4415-4a8d-b5bb-2595b6fcd714 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:34.115543 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d9ebb-2aaa-4bcc-8201-b5a4e065b5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:34.116426 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d9ebb-2aaa-4bcc-8201-b5a4e065b5a6 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:34.116848 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d9ebb-2aaa-4bcc-8201-b5a4e065b5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:34.117274 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d9ebb-2aaa-4bcc-8201-b5a4e065b5a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:34.118307 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d9ebb-2aaa-4bcc-8201-b5a4e065b5a6 None admin] RBAC: Adding query filter params (role.id=dabb7e453c8742aa809ef7543c15d1d3, scope.project.id=cff3a6011f4c4893bddffdad3d69f3df, user.id=9158557f1a2947e2815dc8d586d8b0c3) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:34.118780 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d9ebb-2aaa-4bcc-8201-b5a4e065b5a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:34.120351 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d9ebb-2aaa-4bcc-8201-b5a4e065b5a6 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:34.127044 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 466/912] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:20:33 2022] GET /identity/v3/role_assignments?user.id=9158557f1a2947e2815dc8d586d8b0c3&scope.project.id=cff3a6011f4c4893bddffdad3d69f3df&role.id=dabb7e453c8742aa809ef7543c15d1d3 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:34.543403 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c47af-d5f7-42ed-9b75-84c9222ada45 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:34.543990 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c47af-d5f7-42ed-9b75-84c9222ada45 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:34.543990 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c47af-d5f7-42ed-9b75-84c9222ada45 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:34.545562 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 447/913] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:34.551068 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b31147-293a-407d-b225-1971f2d14b5f None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:34.551357 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b31147-293a-407d-b225-1971f2d14b5f None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:34.551635 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b31147-293a-407d-b225-1971f2d14b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:34.569873 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-98b31147-293a-407d-b225-1971f2d14b5f None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:34.571229 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-98b31147-293a-407d-b225-1971f2d14b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:34.601368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:34.645723 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 467/914] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:20:34.925222 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c47af-d5f7-42ed-9b75-84c9222ada45 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:34.955280 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:35.027812 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4036976-100b-4162-a024-0974aa883c92 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:35.028891 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4036976-100b-4162-a024-0974aa883c92 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:35.028891 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4036976-100b-4162-a024-0974aa883c92 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:35.034354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 468/915] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:20:35.036180 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d234d-f917-4afb-916c-c4acb8aa4c5d None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:35.036533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d234d-f917-4afb-916c-c4acb8aa4c5d None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:35.036883 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d234d-f917-4afb-916c-c4acb8aa4c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:35.047357 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c47af-d5f7-42ed-9b75-84c9222ada45 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:35.059308 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-207d234d-f917-4afb-916c-c4acb8aa4c5d None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:35.060868 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-207d234d-f917-4afb-916c-c4acb8aa4c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:35.084472 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3685c6-9dbf-4dc1-9393-8d644a8735c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:35.085213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3685c6-9dbf-4dc1-9393-8d644a8735c6 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:35.085566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3685c6-9dbf-4dc1-9393-8d644a8735c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:35.085917 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3685c6-9dbf-4dc1-9393-8d644a8735c6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:35.086598 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3685c6-9dbf-4dc1-9393-8d644a8735c6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:35.088164 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3685c6-9dbf-4dc1-9393-8d644a8735c6 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:35.098128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-aa3685c6-9dbf-4dc1-9393-8d644a8735c6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 10:20:35.098128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:35.098128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 10:20:35.098128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 10:20:35.098128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 10:20:35.098128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 10:20:35.098128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 10:20:35.098128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:35.098128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 10:20:35.098128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:35.098128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 10:20:35.098128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:35.098128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:35.098128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:35.098128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:35.098128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:35.099136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:35.099136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:35.099136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:35.099136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:35.099136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:35.099136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:35.099136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:35.099136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 10:20:35.099136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 10:20:35.099136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 10:20:35.099136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 10:20:35.099136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:35.099136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:35.099136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:35.099136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:35.099136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:35.099136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:35.100028 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:35.100028 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:35.100028 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:35.100028 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:35.100028 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:35.100028 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:35.100028 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:35.100028 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:35.100028 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:35.100028 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:35.100028 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 10:20:35.100028 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:35.100028 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 10:20:35.100028 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:35.100028 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 448/916] 10.222.0.49 () {56 vars in 1132 bytes} [Mon Sep 26 10:20:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:35.105008 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3685c6-9dbf-4dc1-9393-8d644a8735c6 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:35.157267 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 469/917] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 10:20:35.214521 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:35.226152 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3685c6-9dbf-4dc1-9393-8d644a8735c6 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:35.265652 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb9881-06ee-4ac0-b4e1-df115c50445f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:35.266604 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb9881-06ee-4ac0-b4e1-df115c50445f None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:35.267094 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb9881-06ee-4ac0-b4e1-df115c50445f None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:35.267520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb9881-06ee-4ac0-b4e1-df115c50445f None admin] PATH_INFO: `/v3/services` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:35.268449 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cb9881-06ee-4ac0-b4e1-df115c50445f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:35.268916 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cb9881-06ee-4ac0-b4e1-df115c50445f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:35.270531 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cb9881-06ee-4ac0-b4e1-df115c50445f None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:35.276778 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 449/918] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Sep 26 10:20:35 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:35.283001 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d234d-f917-4afb-916c-c4acb8aa4c5d None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:35.368522 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32027ead-dc2c-4055-880d-579faa3dcedf None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:35.369148 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32027ead-dc2c-4055-880d-579faa3dcedf None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:35.369928 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32027ead-dc2c-4055-880d-579faa3dcedf None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:35.372169 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 450/919] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:20:35.377055 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece360f4-c4f2-4bfd-b222-b2922e260a84 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:35.377334 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece360f4-c4f2-4bfd-b222-b2922e260a84 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:35.377704 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece360f4-c4f2-4bfd-b222-b2922e260a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:35.384229 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:35.393793 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d234d-f917-4afb-916c-c4acb8aa4c5d None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:35.399413 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-ece360f4-c4f2-4bfd-b222-b2922e260a84 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:35.400859 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-ece360f4-c4f2-4bfd-b222-b2922e260a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:35.429901 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927cb7e-74f1-4699-aa8a-dffb0776a5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:35.430656 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927cb7e-74f1-4699-aa8a-dffb0776a5c5 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:35.431095 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927cb7e-74f1-4699-aa8a-dffb0776a5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:35.431371 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927cb7e-74f1-4699-aa8a-dffb0776a5c5 None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:35.432006 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927cb7e-74f1-4699-aa8a-dffb0776a5c5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:35.432311 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927cb7e-74f1-4699-aa8a-dffb0776a5c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:35.433667 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927cb7e-74f1-4699-aa8a-dffb0776a5c5 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:35.439999 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 470/920] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Sep 26 10:20:35 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:35.440283 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927cb7e-74f1-4699-aa8a-dffb0776a5c5 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:35.480184 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 451/921] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 10:20:35.481339 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece360f4-c4f2-4bfd-b222-b2922e260a84 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:35.567260 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927cb7e-74f1-4699-aa8a-dffb0776a5c5 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:35.594814 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:35.613065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6f5ce-2419-4074-a475-343a873bd638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:35.613690 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6f5ce-2419-4074-a475-343a873bd638 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:35.613965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6f5ce-2419-4074-a475-343a873bd638 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:35.614264 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6f5ce-2419-4074-a475-343a873bd638 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:35.614844 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e6f5ce-2419-4074-a475-343a873bd638 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:35.615633 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece360f4-c4f2-4bfd-b222-b2922e260a84 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:35.616198 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e6f5ce-2419-4074-a475-343a873bd638 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:35.627701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-a7e6f5ce-2419-4074-a475-343a873bd638 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 10:20:35.627701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:35.627701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 10:20:35.627701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 10:20:35.627701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 10:20:35.627701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 10:20:35.627701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 10:20:35.627701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:35.627701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 10:20:35.627701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:35.627701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 10:20:35.627701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:35.627701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:35.627701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:35.627701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:35.627701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:35.627701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:35.628720 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:35.628720 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:35.628720 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:35.628720 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:35.628720 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:35.628720 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:35.628720 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 10:20:35.628720 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 10:20:35.628720 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 10:20:35.628720 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 10:20:35.628720 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:35.628720 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:35.628720 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:35.628720 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:35.628720 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:35.628720 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:35.629563 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:35.629563 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:35.629563 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:35.629563 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:35.629563 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:35.629563 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:35.629563 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:35.629563 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:35.629563 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:35.629563 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:35.629563 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 10:20:35.629563 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:35.629563 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 10:20:35.629563 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:35.634068 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 471/922] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Sep 26 10:20:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:35.635168 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6f5ce-2419-4074-a475-343a873bd638 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:35.663280 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:35.664442 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58059eb8-09c8-495b-9787-ab37a0941c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:35.665721 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58059eb8-09c8-495b-9787-ab37a0941c7f None admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:35.666237 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58059eb8-09c8-495b-9787-ab37a0941c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:35.666933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58059eb8-09c8-495b-9787-ab37a0941c7f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:35.668459 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58059eb8-09c8-495b-9787-ab37a0941c7f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:35.671243 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58059eb8-09c8-495b-9787-ab37a0941c7f None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:35.710427 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 452/923] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:20:35 2022] POST /identity/v3/endpoints => generated 347 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:20:35.748161 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58059eb8-09c8-495b-9787-ab37a0941c7f None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:35.763564 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6f5ce-2419-4074-a475-343a873bd638 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:35.822477 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00daa0c8-88f1-4a4c-80ce-8774c4f66aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:35.823091 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00daa0c8-88f1-4a4c-80ce-8774c4f66aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:35.823362 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00daa0c8-88f1-4a4c-80ce-8774c4f66aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:35.823629 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00daa0c8-88f1-4a4c-80ce-8774c4f66aa1 None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:35.824264 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00daa0c8-88f1-4a4c-80ce-8774c4f66aa1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:35.824558 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00daa0c8-88f1-4a4c-80ce-8774c4f66aa1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:35.825985 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00daa0c8-88f1-4a4c-80ce-8774c4f66aa1 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:35.836642 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 472/924] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Sep 26 10:20:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:35.837716 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00daa0c8-88f1-4a4c-80ce-8774c4f66aa1 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:35.895969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58059eb8-09c8-495b-9787-ab37a0941c7f None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:35.940189 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:35.943979 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421fb1d-a941-42f4-a0d3-7bc88fed4293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:35.944879 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421fb1d-a941-42f4-a0d3-7bc88fed4293 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:35.945383 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421fb1d-a941-42f4-a0d3-7bc88fed4293 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:35.945865 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421fb1d-a941-42f4-a0d3-7bc88fed4293 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:35.946813 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5421fb1d-a941-42f4-a0d3-7bc88fed4293 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:35.948396 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5421fb1d-a941-42f4-a0d3-7bc88fed4293 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:35.960743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-5421fb1d-a941-42f4-a0d3-7bc88fed4293 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 10:20:35.960743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:35.960743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 10:20:35.960743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 10:20:35.960743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 10:20:35.960743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 10:20:35.960743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 10:20:35.960743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:35.960743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 10:20:35.960743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:35.960743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 10:20:35.960743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:35.960743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:35.960743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:35.960743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:35.960743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:35.960743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:35.961852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:35.961852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:35.961852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:35.961852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:35.961852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:35.961852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:35.961852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 10:20:35.961852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 10:20:35.961852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 10:20:35.961852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 10:20:35.961852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:35.961852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:35.961852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:35.961852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:35.961852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:35.961852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:35.962756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:35.962756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:35.962756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:35.962756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:35.962756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:35.962756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:35.962756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:35.962756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:35.962756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:35.962756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:35.962756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 10:20:35.962756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:35.962756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 10:20:35.962756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:35.962756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 453/925] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 10:20:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:20:35.966547 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00daa0c8-88f1-4a4c-80ce-8774c4f66aa1 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:35.967853 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421fb1d-a941-42f4-a0d3-7bc88fed4293 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:35.981070 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:36.022286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f48e0-9832-446f-b678-22af1a6ffcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:36.022900 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f48e0-9832-446f-b678-22af1a6ffcbd None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:36.024080 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f48e0-9832-446f-b678-22af1a6ffcbd None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:36.024383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f48e0-9832-446f-b678-22af1a6ffcbd None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:36.025080 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93f48e0-9832-446f-b678-22af1a6ffcbd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:36.025448 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93f48e0-9832-446f-b678-22af1a6ffcbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:36.026902 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93f48e0-9832-446f-b678-22af1a6ffcbd None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:36.033590 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 473/926] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Sep 26 10:20:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:36.038801 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f48e0-9832-446f-b678-22af1a6ffcbd None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:36.114452 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421fb1d-a941-42f4-a0d3-7bc88fed4293 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:36.164051 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea8456-577f-4dff-a265-9173ab4c8a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:36.165142 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea8456-577f-4dff-a265-9173ab4c8a33 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:36.165589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea8456-577f-4dff-a265-9173ab4c8a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:36.166046 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea8456-577f-4dff-a265-9173ab4c8a33 None admin] PATH_INFO: `/v3/services` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:36.167004 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ea8456-577f-4dff-a265-9173ab4c8a33 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:36.167121 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f48e0-9832-446f-b678-22af1a6ffcbd None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:36.167557 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ea8456-577f-4dff-a265-9173ab4c8a33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:36.169923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ea8456-577f-4dff-a265-9173ab4c8a33 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:36.181577 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 454/927] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 10:20:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:36.183660 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea8456-577f-4dff-a265-9173ab4c8a33 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:36.211422 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceabe79e-f433-42c5-b175-ff4a32ea21c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:36.212038 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabe79e-f433-42c5-b175-ff4a32ea21c6 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:36.212359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabe79e-f433-42c5-b175-ff4a32ea21c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:36.212638 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabe79e-f433-42c5-b175-ff4a32ea21c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:36.213341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceabe79e-f433-42c5-b175-ff4a32ea21c6 None admin] RBAC: Adding query filter params (interface=public, service_id=2e7c9b695cab465c8127d3715e5cc9fe, region_id=RegionOne) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:36.213649 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceabe79e-f433-42c5-b175-ff4a32ea21c6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:36.215496 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceabe79e-f433-42c5-b175-ff4a32ea21c6 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:36.221687 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 474/928] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Sep 26 10:20:36 2022] GET /identity/v3/endpoints?service_id=2e7c9b695cab465c8127d3715e5cc9fe&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:36.226810 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceabe79e-f433-42c5-b175-ff4a32ea21c6 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:36.240098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:36.277733 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:36.328499 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea8456-577f-4dff-a265-9173ab4c8a33 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:36.360846 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceabe79e-f433-42c5-b175-ff4a32ea21c6 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:36.376629 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b091c3-c175-4a05-946c-29275235bf92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:36.377495 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b091c3-c175-4a05-946c-29275235bf92 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:36.378281 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b091c3-c175-4a05-946c-29275235bf92 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:36.378572 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b091c3-c175-4a05-946c-29275235bf92 None admin] PATH_INFO: `/v3/services` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:36.379489 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b091c3-c175-4a05-946c-29275235bf92 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:36.379794 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b091c3-c175-4a05-946c-29275235bf92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:36.381847 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b091c3-c175-4a05-946c-29275235bf92 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:36.390186 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 455/929] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Sep 26 10:20:36 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:36.391666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e67fb-597a-463e-ac9a-0d79a7fc9b3b None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:36.391945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e67fb-597a-463e-ac9a-0d79a7fc9b3b None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:36.392239 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e67fb-597a-463e-ac9a-0d79a7fc9b3b None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:36.393709 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 456/930] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:20:36.394518 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e67fb-597a-463e-ac9a-0d79a7fc9b3b None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:36.409075 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f557f-1c09-4c1a-bc77-490909271fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:36.409804 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f557f-1c09-4c1a-bc77-490909271fef None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:36.410113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f557f-1c09-4c1a-bc77-490909271fef None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:36.410407 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f557f-1c09-4c1a-bc77-490909271fef None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:36.414114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f557f-1c09-4c1a-bc77-490909271fef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:36.414114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f557f-1c09-4c1a-bc77-490909271fef None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:36.423415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 475/931] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Sep 26 10:20:36 2022] GET /identity/v3/services => generated 1497 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:20:36.424684 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81b41f-e2b1-4d82-8460-701e2087b492 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:36.424973 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81b41f-e2b1-4d82-8460-701e2087b492 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:36.425246 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81b41f-e2b1-4d82-8460-701e2087b492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:36.448118 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-0c81b41f-e2b1-4d82-8460-701e2087b492 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:36.449094 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-0c81b41f-e2b1-4d82-8460-701e2087b492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:36.535231 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 476/932] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:20:36.547907 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e67fb-597a-463e-ac9a-0d79a7fc9b3b None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:36.587576 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee87bd-32a2-4c30-ae3d-2c3897a3b1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:36.588202 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee87bd-32a2-4c30-ae3d-2c3897a3b1a0 None admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:36.588691 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee87bd-32a2-4c30-ae3d-2c3897a3b1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:36.589198 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee87bd-32a2-4c30-ae3d-2c3897a3b1a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:36.590510 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfee87bd-32a2-4c30-ae3d-2c3897a3b1a0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:36.592641 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfee87bd-32a2-4c30-ae3d-2c3897a3b1a0 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:36.599622 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:36.621081 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 457/933] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:20:36 2022] POST /identity/v3/endpoints => generated 346 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:20:36.825148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81b41f-e2b1-4d82-8460-701e2087b492 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:36.827875 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:36.934052 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81b41f-e2b1-4d82-8460-701e2087b492 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:36.979511 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4faf6-5ed4-4878-bb72-e547e8aa7b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:36.980152 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4faf6-5ed4-4878-bb72-e547e8aa7b90 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:36.980432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4faf6-5ed4-4878-bb72-e547e8aa7b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:36.980706 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4faf6-5ed4-4878-bb72-e547e8aa7b90 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:36.991533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be4faf6-5ed4-4878-bb72-e547e8aa7b90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:20:36.996331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-8be4faf6-5ed4-4878-bb72-e547e8aa7b90 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 10:20:36.996331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:36.996331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:36.996331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:36.996331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:36.996331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:36.996331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:36.996331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:36.996331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:36.996331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:36.996331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:36.996331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:36.996331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:20:36.996331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:20:36.996331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:20:36.996331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:20:36.997298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:20:36.997298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:20:36.997298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:20:36.997298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:20:36.997298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:36.997298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:36.997298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:36.997298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:36.997298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:36.997298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:36.997298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:36.997298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:36.997298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:36.997298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:36.997298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:36.997298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:36.998195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:36.998195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:36.998195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:36.998195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:36.998195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:20:36.998195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:20:36.998195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:20:36.998195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:20:36.998195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:20:36.998195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:20:36.998195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 10:20:36.998195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:36.998195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 477/934] 10.222.0.49 () {56 vars in 1120 bytes} [Mon Sep 26 10:20:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:37.005445 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee87bd-32a2-4c30-ae3d-2c3897a3b1a0 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:37.111198 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee87bd-32a2-4c30-ae3d-2c3897a3b1a0 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:37.147039 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d283a1e-b280-4ea4-86e0-f66bad085960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:37.148002 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d283a1e-b280-4ea4-86e0-f66bad085960 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:37.148481 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d283a1e-b280-4ea4-86e0-f66bad085960 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:37.148969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d283a1e-b280-4ea4-86e0-f66bad085960 None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:37.149846 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d283a1e-b280-4ea4-86e0-f66bad085960 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:37.150368 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d283a1e-b280-4ea4-86e0-f66bad085960 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:37.151963 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d283a1e-b280-4ea4-86e0-f66bad085960 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:37.160786 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 458/935] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Sep 26 10:20:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:37.166677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4faf6-5ed4-4878-bb72-e547e8aa7b90 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:37.231378 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:37.274927 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4faf6-5ed4-4878-bb72-e547e8aa7b90 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:37.309463 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b3ab7-a2f2-4600-9287-20b50032de5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:37.310641 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b3ab7-a2f2-4600-9287-20b50032de5c None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:37.310921 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b3ab7-a2f2-4600-9287-20b50032de5c None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:37.311195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b3ab7-a2f2-4600-9287-20b50032de5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:37.321453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206b3ab7-a2f2-4600-9287-20b50032de5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:37.322937 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206b3ab7-a2f2-4600-9287-20b50032de5c None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:37.338689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-206b3ab7-a2f2-4600-9287-20b50032de5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:37.338689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:37.338689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:37.338689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:37.338689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:37.338689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:37.338689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:37.338689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:37.338689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:37.338689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:37.338689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:37.338689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:37.338689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:37.338689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:37.338689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:37.338689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:37.339861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:37.339861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:37.339861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:37.339861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:37.339861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:37.339861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:37.339861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:37.339861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:37.339861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:37.339861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:37.339861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:37.339861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:37.339861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:37.339861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:37.339861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:37.339861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:37.339861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:37.340980 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:37.340980 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:37.340980 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:37.342992 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 478/936] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:37.348476 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d283a1e-b280-4ea4-86e0-f66bad085960 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:37.416527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:37.471334 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d283a1e-b280-4ea4-86e0-f66bad085960 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:37.506490 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c0104-b32e-40b5-9368-16c8071f26f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:37.507094 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c0104-b32e-40b5-9368-16c8071f26f1 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:37.507367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c0104-b32e-40b5-9368-16c8071f26f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:37.507635 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c0104-b32e-40b5-9368-16c8071f26f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:37.508277 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170c0104-b32e-40b5-9368-16c8071f26f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:37.508572 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170c0104-b32e-40b5-9368-16c8071f26f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:37.509939 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170c0104-b32e-40b5-9368-16c8071f26f1 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:37.522952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 459/937] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:37.528282 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b3ab7-a2f2-4600-9287-20b50032de5c None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:37.640988 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b3ab7-a2f2-4600-9287-20b50032de5c None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:37.652162 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:37.676746 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee84b9-8eeb-49e4-8087-93129dfc6c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:37.677302 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee84b9-8eeb-49e4-8087-93129dfc6c5f None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:37.677584 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee84b9-8eeb-49e4-8087-93129dfc6c5f None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:37.677861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee84b9-8eeb-49e4-8087-93129dfc6c5f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:37.685998 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ee84b9-8eeb-49e4-8087-93129dfc6c5f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:37.687457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ee84b9-8eeb-49e4-8087-93129dfc6c5f None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:37.698578 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-a2ee84b9-8eeb-49e4-8087-93129dfc6c5f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 10:20:37.698578 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:37.698578 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:37.698578 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:37.698578 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:37.698578 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:37.698578 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:37.698578 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:37.698578 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:37.698578 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:37.698578 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:37.698578 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:37.698578 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:20:37.698578 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:20:37.698578 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:20:37.698578 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:20:37.699504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:37.699504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:37.699504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:20:37.699504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:37.699504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:20:37.699504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:37.699504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:37.699504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:37.699504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:37.699504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:37.699504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:37.699504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:37.699504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:37.699504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:37.699504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:37.699504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:37.699504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:37.699504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:37.700487 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:37.700487 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:37.700487 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:20:37.700487 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:20:37.700487 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:20:37.700487 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:20:37.700487 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:20:37.700487 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:20:37.700487 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 10:20:37.700487 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:37.700487 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 479/938] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Sep 26 10:20:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:37.707729 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c0104-b32e-40b5-9368-16c8071f26f1 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:37.817623 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c0104-b32e-40b5-9368-16c8071f26f1 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:37.831675 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:37.854237 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c7195-df22-41bb-8f27-1cd84827416c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:37.855154 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c7195-df22-41bb-8f27-1cd84827416c None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:37.855638 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c7195-df22-41bb-8f27-1cd84827416c None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:37.856167 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c7195-df22-41bb-8f27-1cd84827416c None admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:37.857687 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c7195-df22-41bb-8f27-1cd84827416c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:37.858217 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c7195-df22-41bb-8f27-1cd84827416c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:37.859844 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c7195-df22-41bb-8f27-1cd84827416c None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:37.880435 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 460/939] 10.222.0.49 () {56 vars in 1149 bytes} [Mon Sep 26 10:20:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:37.886177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee84b9-8eeb-49e4-8087-93129dfc6c5f None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:37.995923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2d1f6-905c-4f10-b5ef-fa38ff173ce1 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:37.996097 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2d1f6-905c-4f10-b5ef-fa38ff173ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:37.996364 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2d1f6-905c-4f10-b5ef-fa38ff173ce1 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:37.998538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 461/940] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:20:37.999759 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee84b9-8eeb-49e4-8087-93129dfc6c5f None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:38.003377 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24293a5c-6aa1-4b02-b156-3202feb0c759 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:38.003657 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24293a5c-6aa1-4b02-b156-3202feb0c759 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:38.003927 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24293a5c-6aa1-4b02-b156-3202feb0c759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:38.023059 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-24293a5c-6aa1-4b02-b156-3202feb0c759 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:38.024546 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-24293a5c-6aa1-4b02-b156-3202feb0c759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:38.041592 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598612e2-1cbd-4b7a-8e70-fe330db18c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:38.042269 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598612e2-1cbd-4b7a-8e70-fe330db18c25 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:38.042545 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598612e2-1cbd-4b7a-8e70-fe330db18c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:38.042815 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598612e2-1cbd-4b7a-8e70-fe330db18c25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:38.055199 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598612e2-1cbd-4b7a-8e70-fe330db18c25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:38.056927 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598612e2-1cbd-4b7a-8e70-fe330db18c25 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:38.080542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-598612e2-1cbd-4b7a-8e70-fe330db18c25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:38.080542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:38.080542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:38.080542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:38.080542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:38.080542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:38.080542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:38.080542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:38.080542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:38.080542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:38.080542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:38.080542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:38.080542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:38.080542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:38.080542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:38.080542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:38.081573 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:38.081573 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:38.081573 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:38.081573 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:38.081573 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:38.081573 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:38.081573 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:38.081573 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:38.081573 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:38.081573 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:38.081573 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:38.081573 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:38.081573 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:38.081573 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:38.081573 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:38.081573 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:38.081573 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:38.083134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:38.083134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:38.083134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:38.083134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 480/941] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:38.087618 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598612e2-1cbd-4b7a-8e70-fe330db18c25 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:38.113441 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 462/942] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 10:20:38.122611 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:38.198839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598612e2-1cbd-4b7a-8e70-fe330db18c25 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:38.234163 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9ea4b-abda-4971-ba30-9114a78b2773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:38.234770 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9ea4b-abda-4971-ba30-9114a78b2773 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:38.235046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9ea4b-abda-4971-ba30-9114a78b2773 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:38.235315 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9ea4b-abda-4971-ba30-9114a78b2773 None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:38.235951 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a9ea4b-abda-4971-ba30-9114a78b2773 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:38.236248 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a9ea4b-abda-4971-ba30-9114a78b2773 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:38.237599 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a9ea4b-abda-4971-ba30-9114a78b2773 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:38.250639 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 481/943] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:38.256162 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24293a5c-6aa1-4b02-b156-3202feb0c759 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:38.361905 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:38.365517 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24293a5c-6aa1-4b02-b156-3202feb0c759 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:38.388599 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9ea4b-abda-4971-ba30-9114a78b2773 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:38.405408 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de2348-27ed-4142-a5df-345117b217b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:38.406544 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de2348-27ed-4142-a5df-345117b217b5 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:38.406826 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de2348-27ed-4142-a5df-345117b217b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:38.407375 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de2348-27ed-4142-a5df-345117b217b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:38.415710 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40de2348-27ed-4142-a5df-345117b217b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:38.418765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40de2348-27ed-4142-a5df-345117b217b5 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:38.434379 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-40de2348-27ed-4142-a5df-345117b217b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:38.434379 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:38.434379 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:38.434379 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:38.434379 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:38.434379 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:38.434379 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:38.434379 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:38.434379 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:38.434379 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:38.434379 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:38.434379 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:38.434379 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:20:38.434379 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:20:38.434379 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:20:38.435907 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:20:38.435907 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:38.435907 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:38.435907 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:38.435907 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:38.435907 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:38.435907 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:38.435907 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:38.435907 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:38.435907 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:38.435907 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:38.435907 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:38.435907 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:38.435907 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:38.435907 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:38.435907 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:38.435907 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:38.436952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:20:38.436952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:20:38.436952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:20:38.436952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:20:38.436952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:20:38.436952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:20:38.436952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:38.436952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:38.436952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 463/944] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:20:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:38.443554 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de2348-27ed-4142-a5df-345117b217b5 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:38.493790 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:38.515845 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9ea4b-abda-4971-ba30-9114a78b2773 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:38.554601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112bf8f6-8c98-4ffe-a399-f5e0b755f4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:38.555236 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112bf8f6-8c98-4ffe-a399-f5e0b755f4ed None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:38.555616 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112bf8f6-8c98-4ffe-a399-f5e0b755f4ed None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:38.555887 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112bf8f6-8c98-4ffe-a399-f5e0b755f4ed None admin] PATH_INFO: `/v3/services/compute` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:38.556473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112bf8f6-8c98-4ffe-a399-f5e0b755f4ed None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:38.557840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112bf8f6-8c98-4ffe-a399-f5e0b755f4ed None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:38.566106 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de2348-27ed-4142-a5df-345117b217b5 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:38.567505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-112bf8f6-8c98-4ffe-a399-f5e0b755f4ed None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 10:20:38.567505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:38.567505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 10:20:38.567505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 10:20:38.567505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 10:20:38.567505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 10:20:38.567505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 10:20:38.567505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:38.567505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 10:20:38.567505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:38.567505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 10:20:38.567505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:38.567505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:38.567505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:38.567505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:38.567505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:38.567505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:38.568469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:38.568469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:38.568469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:38.568469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:38.568469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:38.568469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:38.568469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 10:20:38.568469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 10:20:38.568469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 10:20:38.568469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 10:20:38.568469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:38.568469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:38.568469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:38.568469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:38.568469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:38.568469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:38.569326 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:38.569326 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:38.569326 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:38.569326 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:38.569326 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:38.569326 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:38.569326 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:38.569326 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:38.569326 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:38.569326 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:38.569326 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 10:20:38.569326 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:38.569326 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 10:20:38.569326 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:38.573751 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 482/945] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:20:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:20:38.574823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112bf8f6-8c98-4ffe-a399-f5e0b755f4ed None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:38.623206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a616c-cde7-4aaf-aa36-bec894d417de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:38.624271 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a616c-cde7-4aaf-aa36-bec894d417de None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:38.624731 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a616c-cde7-4aaf-aa36-bec894d417de None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:38.625406 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a616c-cde7-4aaf-aa36-bec894d417de None admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:38.627320 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a616c-cde7-4aaf-aa36-bec894d417de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:38.627780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a616c-cde7-4aaf-aa36-bec894d417de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:38.629387 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a616c-cde7-4aaf-aa36-bec894d417de None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:38.651695 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 464/946] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:20:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:38.652797 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a616c-cde7-4aaf-aa36-bec894d417de None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:38.683901 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:38.706045 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112bf8f6-8c98-4ffe-a399-f5e0b755f4ed None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:38.750433 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1122b-f2fc-463a-9e85-4698f5995a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:38.751691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1122b-f2fc-463a-9e85-4698f5995a9e None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:38.752431 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1122b-f2fc-463a-9e85-4698f5995a9e None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:38.752901 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1122b-f2fc-463a-9e85-4698f5995a9e None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:38.753707 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f1122b-f2fc-463a-9e85-4698f5995a9e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:38.754177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f1122b-f2fc-463a-9e85-4698f5995a9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:38.755703 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f1122b-f2fc-463a-9e85-4698f5995a9e None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:38.762324 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 483/947] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Sep 26 10:20:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:38.764125 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0bec2-08eb-4883-b567-3eaaf6f75358 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:38.764399 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0bec2-08eb-4883-b567-3eaaf6f75358 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:38.764666 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0bec2-08eb-4883-b567-3eaaf6f75358 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:38.766276 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 484/948] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:38.771407 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9348f-c016-4cad-a557-6ec26ab5ab74 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:38.771687 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9348f-c016-4cad-a557-6ec26ab5ab74 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:38.771956 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9348f-c016-4cad-a557-6ec26ab5ab74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:38.784388 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a616c-cde7-4aaf-aa36-bec894d417de None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:38.795327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-d2a9348f-c016-4cad-a557-6ec26ab5ab74 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:38.796649 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-d2a9348f-c016-4cad-a557-6ec26ab5ab74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:38.800544 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:38.824618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37929fd4-0930-4116-a6a7-b8111c3960ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:38.825353 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37929fd4-0930-4116-a6a7-b8111c3960ee None admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:38.825633 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37929fd4-0930-4116-a6a7-b8111c3960ee None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:38.825841 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37929fd4-0930-4116-a6a7-b8111c3960ee None admin] PATH_INFO: `/v3/projects/cff3a6011f4c4893bddffdad3d69f3df/users/9158557f1a2947e2815dc8d586d8b0c3/roles/dabb7e453c8742aa809ef7543c15d1d3` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:38.828088 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37929fd4-0930-4116-a6a7-b8111c3960ee None admin] RBAC: Authorizing `identity:create_grant(project_id=cff3a6011f4c4893bddffdad3d69f3df, user_id=9158557f1a2947e2815dc8d586d8b0c3, role_id=dabb7e453c8742aa809ef7543c15d1d3)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:38.829509 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37929fd4-0930-4116-a6a7-b8111c3960ee None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:38.844766 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 465/949] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 10:20:38 2022] PUT /identity/v3/projects/cff3a6011f4c4893bddffdad3d69f3df/users/9158557f1a2947e2815dc8d586d8b0c3/roles/dabb7e453c8742aa809ef7543c15d1d3 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:20:38.845428 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37929fd4-0930-4116-a6a7-b8111c3960ee None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:38.902438 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 485/950] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 10:20:38.958068 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:38.987278 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37929fd4-0930-4116-a6a7-b8111c3960ee None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:39.030855 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ead04c-aecd-4ffa-a55d-0634c2b29693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:39.031860 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ead04c-aecd-4ffa-a55d-0634c2b29693 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:39.032337 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ead04c-aecd-4ffa-a55d-0634c2b29693 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:39.032824 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ead04c-aecd-4ffa-a55d-0634c2b29693 None admin] PATH_INFO: `/v3/services` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:39.033682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ead04c-aecd-4ffa-a55d-0634c2b29693 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:39.034234 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ead04c-aecd-4ffa-a55d-0634c2b29693 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:39.035890 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ead04c-aecd-4ffa-a55d-0634c2b29693 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:39.041928 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 466/951] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Sep 26 10:20:38 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:39.056868 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2eb17-577b-4e1a-a671-de7bb6661b04 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:39.057232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2eb17-577b-4e1a-a671-de7bb6661b04 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:39.057432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2eb17-577b-4e1a-a671-de7bb6661b04 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:39.060125 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 486/952] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:39.065301 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f242b44-d356-4ded-bdff-6d4451cb922a None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:39.065582 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f242b44-d356-4ded-bdff-6d4451cb922a None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:39.065856 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f242b44-d356-4ded-bdff-6d4451cb922a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:39.086597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-3f242b44-d356-4ded-bdff-6d4451cb922a None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:39.087839 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-3f242b44-d356-4ded-bdff-6d4451cb922a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:39.171698 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 467/953] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:20:39.194384 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2eb17-577b-4e1a-a671-de7bb6661b04 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:39.310720 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2eb17-577b-4e1a-a671-de7bb6661b04 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:39.348444 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e866c5-0b16-4375-b20d-493cd81654b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:39.349346 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e866c5-0b16-4375-b20d-493cd81654b7 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:39.349778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e866c5-0b16-4375-b20d-493cd81654b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:39.350218 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e866c5-0b16-4375-b20d-493cd81654b7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:39.351052 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e866c5-0b16-4375-b20d-493cd81654b7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:39.352551 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e866c5-0b16-4375-b20d-493cd81654b7 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:39.362164 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-17e866c5-0b16-4375-b20d-493cd81654b7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 10:20:39.362164 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:39.362164 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 10:20:39.362164 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 10:20:39.362164 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 10:20:39.362164 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 10:20:39.362164 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 10:20:39.362164 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:39.362164 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 10:20:39.362164 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:39.362164 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 10:20:39.362164 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:39.362164 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:39.362164 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:39.362164 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:39.362164 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:39.362164 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:39.363381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:39.363381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:39.363381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:39.363381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:39.363381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:39.363381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:39.363381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 10:20:39.363381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 10:20:39.363381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 10:20:39.363381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 10:20:39.363381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:39.363381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:39.363381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:39.363381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:39.363381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:39.363381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:39.364491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:39.364491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:39.364491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:39.364491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:39.364491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:39.364491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:39.364491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:39.364491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:39.364491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:39.364491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:39.364491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 10:20:39.364491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:39.364491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 10:20:39.364491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:39.367018 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 487/954] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Sep 26 10:20:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:39.375963 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f242b44-d356-4ded-bdff-6d4451cb922a None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:39.443621 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e866c5-0b16-4375-b20d-493cd81654b7 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:39.448009 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:39.468263 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:39.497612 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f242b44-d356-4ded-bdff-6d4451cb922a None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:39.542520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e3509-9095-4947-90d9-5dbc12d12350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:39.543666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e3509-9095-4947-90d9-5dbc12d12350 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:39.543962 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e3509-9095-4947-90d9-5dbc12d12350 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:39.544231 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e3509-9095-4947-90d9-5dbc12d12350 None admin] PATH_INFO: `/v3/services` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:39.545304 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e3509-9095-4947-90d9-5dbc12d12350 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:39.545677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e3509-9095-4947-90d9-5dbc12d12350 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:39.547920 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e3509-9095-4947-90d9-5dbc12d12350 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:39.555080 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 468/955] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Sep 26 10:20:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:39.560684 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e3509-9095-4947-90d9-5dbc12d12350 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:39.581026 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e866c5-0b16-4375-b20d-493cd81654b7 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:39.634206 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54850af-20f9-490e-86e6-36f0dc5c8f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:39.635185 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54850af-20f9-490e-86e6-36f0dc5c8f94 None admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:39.635699 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54850af-20f9-490e-86e6-36f0dc5c8f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:39.636176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54850af-20f9-490e-86e6-36f0dc5c8f94 None admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:39.637274 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54850af-20f9-490e-86e6-36f0dc5c8f94 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:39.640123 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54850af-20f9-490e-86e6-36f0dc5c8f94 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:39.643416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: WARNING keystone.server.flask.common [None req-c54850af-20f9-490e-86e6-36f0dc5c8f94 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 26 10:20:39.666264 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.sql.core [None req-c54850af-20f9-490e-86e6-36f0dc5c8f94 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 26 10:20:39.666264 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 26 10:20:39.666264 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [parameters: {'id': '911d468eef434eaa89114b14bee35e57', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 26 10:20:39.666264 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81420) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 26 10:20:39.677672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e3509-9095-4947-90d9-5dbc12d12350 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:39.686542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-c54850af-20f9-490e-86e6-36f0dc5c8f94 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 911d468eef434eaa89114b14bee35e57.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 911d468eef434eaa89114b14bee35e57. Sep 26 10:20:39.686542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:39.686542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 26 10:20:39.686542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 26 10:20:39.686542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 26 10:20:39.686542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 26 10:20:39.686542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 26 10:20:39.686542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application result = self._query(query) Sep 26 10:20:39.686542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 26 10:20:39.686542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application conn.query(q) Sep 26 10:20:39.686542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 26 10:20:39.686542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 26 10:20:39.686542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 26 10:20:39.687416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application result.read() Sep 26 10:20:39.687416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 26 10:20:39.687416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 26 10:20:39.687416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 26 10:20:39.687416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 26 10:20:39.687416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 26 10:20:39.687416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 26 10:20:39.687416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 26 10:20:39.687416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 26 10:20:39.687416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 26 10:20:39.687416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:39.687416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 26 10:20:39.687416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:39.687416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:39.687416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 26 10:20:39.687416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 26 10:20:39.687416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 26 10:20:39.687416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 26 10:20:39.688400 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 26 10:20:39.688400 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application next(self.gen) Sep 26 10:20:39.688400 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 26 10:20:39.688400 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application yield resource Sep 26 10:20:39.688400 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 26 10:20:39.688400 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application next(self.gen) Sep 26 10:20:39.688400 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 26 10:20:39.688400 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self.session.rollback() Sep 26 10:20:39.688400 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 26 10:20:39.688400 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self.force_reraise() Sep 26 10:20:39.688400 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 26 10:20:39.688400 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise self.value Sep 26 10:20:39.688400 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 26 10:20:39.688400 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 26 10:20:39.688400 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 26 10:20:39.688400 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application session.commit() Sep 26 10:20:39.688400 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 26 10:20:39.689298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 26 10:20:39.689298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 26 10:20:39.689298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._prepare_impl() Sep 26 10:20:39.689298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 26 10:20:39.689298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self.session.flush() Sep 26 10:20:39.689298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 26 10:20:39.689298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._flush(objects) Sep 26 10:20:39.689298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 26 10:20:39.689298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 26 10:20:39.689298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 26 10:20:39.689298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application compat.raise_( Sep 26 10:20:39.689298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 26 10:20:39.689298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception Sep 26 10:20:39.689298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 26 10:20:39.689298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application flush_context.execute() Sep 26 10:20:39.689298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 26 10:20:39.689298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rec.execute(self) Sep 26 10:20:39.690406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 26 10:20:39.690406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 26 10:20:39.690406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 26 10:20:39.690406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 26 10:20:39.690406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 26 10:20:39.690406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 26 10:20:39.690406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 26 10:20:39.690406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 26 10:20:39.690406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 26 10:20:39.690406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 26 10:20:39.690406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 26 10:20:39.690406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 26 10:20:39.690406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 26 10:20:39.690406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 26 10:20:39.690406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 26 10:20:39.691208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 26 10:20:39.691208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 26 10:20:39.691208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception Sep 26 10:20:39.691208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 26 10:20:39.691208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 26 10:20:39.691208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 26 10:20:39.691208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 26 10:20:39.691208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 26 10:20:39.691208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application result = self._query(query) Sep 26 10:20:39.691208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 26 10:20:39.691208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application conn.query(q) Sep 26 10:20:39.691208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 26 10:20:39.691208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 26 10:20:39.691208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 26 10:20:39.691208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application result.read() Sep 26 10:20:39.691208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 26 10:20:39.691208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 26 10:20:39.692098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 26 10:20:39.692098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 26 10:20:39.692098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 26 10:20:39.692098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 26 10:20:39.692098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 26 10:20:39.692098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 26 10:20:39.692098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 26 10:20:39.692098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 26 10:20:39.692098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [parameters: {'id': '911d468eef434eaa89114b14bee35e57', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 26 10:20:39.692098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 26 10:20:39.692098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:39.692098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 10:20:39.692098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:39.692098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:39.692098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 26 10:20:39.693143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 26 10:20:39.693143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 26 10:20:39.693143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 26 10:20:39.693143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 26 10:20:39.693143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:39.693143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 10:20:39.693143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:39.693143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:39.693143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:39.693143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:39.693143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:39.693143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:39.693143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:39.693143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:39.693143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:39.693143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:39.693143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:39.695203 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:39.695203 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 26 10:20:39.695203 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 26 10:20:39.695203 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:39.695203 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:39.695203 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 26 10:20:39.695203 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 26 10:20:39.695203 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 911d468eef434eaa89114b14bee35e57. Sep 26 10:20:39.695203 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:39.698915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 488/956] 10.222.0.49 () {60 vars in 1140 bytes} [Mon Sep 26 10:20:39 2022] POST /identity/v3/projects => generated 262 bytes in 256 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:20:39.711434 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54850af-20f9-490e-86e6-36f0dc5c8f94 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:39.722713 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302e293-3491-4517-ba40-d20d12718e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:39.723841 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302e293-3491-4517-ba40-d20d12718e33 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:39.724380 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302e293-3491-4517-ba40-d20d12718e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:39.724847 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302e293-3491-4517-ba40-d20d12718e33 None admin] PATH_INFO: `/v3/services` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:39.725798 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302e293-3491-4517-ba40-d20d12718e33 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:39.726367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302e293-3491-4517-ba40-d20d12718e33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:39.728080 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302e293-3491-4517-ba40-d20d12718e33 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:39.738161 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 469/957] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Sep 26 10:20:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:39.739793 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302e293-3491-4517-ba40-d20d12718e33 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:39.744435 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:39.797899 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:39.864555 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54850af-20f9-490e-86e6-36f0dc5c8f94 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:39.882171 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302e293-3491-4517-ba40-d20d12718e33 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:39.916567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9205730f-4cc4-498d-ba89-666b8cd439f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:39.917512 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205730f-4cc4-498d-ba89-666b8cd439f1 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:39.918005 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205730f-4cc4-498d-ba89-666b8cd439f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:39.918474 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205730f-4cc4-498d-ba89-666b8cd439f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:39.924869 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5123d03-263b-48b8-80af-3ca7fee6494d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:39.925923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5123d03-263b-48b8-80af-3ca7fee6494d None admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:39.926386 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5123d03-263b-48b8-80af-3ca7fee6494d None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:39.926964 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5123d03-263b-48b8-80af-3ca7fee6494d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:39.927875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5123d03-263b-48b8-80af-3ca7fee6494d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:39.929398 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5123d03-263b-48b8-80af-3ca7fee6494d None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:39.931396 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9205730f-4cc4-498d-ba89-666b8cd439f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:39.933260 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9205730f-4cc4-498d-ba89-666b8cd439f1 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:39.947746 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-9205730f-4cc4-498d-ba89-666b8cd439f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:39.947746 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:39.947746 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:39.947746 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:39.947746 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:39.947746 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:39.947746 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:39.947746 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:39.947746 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:39.947746 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:39.947746 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:39.947746 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:39.947746 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:20:39.947746 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:20:39.947746 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:20:39.949415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:20:39.949415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:39.949415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:39.949415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:39.949415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:39.949415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:39.949415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:39.949415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:39.949415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:39.949415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:39.949415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:39.949415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:39.949415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:39.949415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:39.949415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:39.949415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:39.949415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:39.951176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:20:39.951176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:20:39.951176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:20:39.951176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:20:39.951176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:20:39.951176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:20:39.951176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:39.951176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:39.956875 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 489/958] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:20:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:39.959174 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9205730f-4cc4-498d-ba89-666b8cd439f1 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:39.970200 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 470/959] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:20:39 2022] POST /identity/v3/endpoints => generated 349 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:20:40.077048 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9205730f-4cc4-498d-ba89-666b8cd439f1 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:40.100092 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:40.126820 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbcaa42-2d39-4a03-867f-e9b4246bb747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:40.127347 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbcaa42-2d39-4a03-867f-e9b4246bb747 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:40.127624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbcaa42-2d39-4a03-867f-e9b4246bb747 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:40.127896 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbcaa42-2d39-4a03-867f-e9b4246bb747 None admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:40.128570 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbcaa42-2d39-4a03-867f-e9b4246bb747 None admin] RBAC: Adding query filter params (name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:40.128868 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbcaa42-2d39-4a03-867f-e9b4246bb747 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:40.130433 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbcaa42-2d39-4a03-867f-e9b4246bb747 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:40.158185 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 490/960] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Sep 26 10:20:39 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:41.230816 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efe37d-c5ab-46ac-b138-70202dc4db07 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:41.231851 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efe37d-c5ab-46ac-b138-70202dc4db07 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:41.232291 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efe37d-c5ab-46ac-b138-70202dc4db07 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:41.233846 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 471/961] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:41.239350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a502ef3-5472-4d03-b7bd-d38de3fa026c None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:41.239769 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a502ef3-5472-4d03-b7bd-d38de3fa026c None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:41.239944 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a502ef3-5472-4d03-b7bd-d38de3fa026c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:41.255857 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386329bd-8eaa-4ec6-b78c-9f89720eb32b None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:41.256145 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386329bd-8eaa-4ec6-b78c-9f89720eb32b None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:41.256394 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386329bd-8eaa-4ec6-b78c-9f89720eb32b None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:41.257948 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 472/962] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:20:41.259868 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-4a502ef3-5472-4d03-b7bd-d38de3fa026c None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:41.261412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-4a502ef3-5472-4d03-b7bd-d38de3fa026c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:41.263332 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecedff6-dcd6-4151-b5c6-d84b44b0ffcc None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:41.263683 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecedff6-dcd6-4151-b5c6-d84b44b0ffcc None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:41.264015 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecedff6-dcd6-4151-b5c6-d84b44b0ffcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:41.281722 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-7ecedff6-dcd6-4151-b5c6-d84b44b0ffcc None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:41.283029 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-7ecedff6-dcd6-4151-b5c6-d84b44b0ffcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:41.337198 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 491/963] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 10:20:41.349193 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:41.367484 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 473/964] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:20:41.610004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a502ef3-5472-4d03-b7bd-d38de3fa026c None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:41.641070 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecedff6-dcd6-4151-b5c6-d84b44b0ffcc None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:41.736171 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a502ef3-5472-4d03-b7bd-d38de3fa026c None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:41.743710 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecedff6-dcd6-4151-b5c6-d84b44b0ffcc None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:41.748145 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:41.780719 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55117746-155d-4cb7-84d7-076624b2d623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:41.781577 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55117746-155d-4cb7-84d7-076624b2d623 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:41.781954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55117746-155d-4cb7-84d7-076624b2d623 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:41.782321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55117746-155d-4cb7-84d7-076624b2d623 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:41.785346 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e5b3a2-0ee2-4f92-a04f-6072175c3482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:41.786198 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e5b3a2-0ee2-4f92-a04f-6072175c3482 None admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:41.786676 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e5b3a2-0ee2-4f92-a04f-6072175c3482 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:41.787125 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e5b3a2-0ee2-4f92-a04f-6072175c3482 None admin] PATH_INFO: `/v3/services` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:41.787954 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e5b3a2-0ee2-4f92-a04f-6072175c3482 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:41.789511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e5b3a2-0ee2-4f92-a04f-6072175c3482 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:41.793609 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55117746-155d-4cb7-84d7-076624b2d623 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:20:41.798778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-55117746-155d-4cb7-84d7-076624b2d623 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 10:20:41.798778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:41.798778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:41.798778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:41.798778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:41.798778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:41.798778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:41.798778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:41.798778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:41.798778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:41.798778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:41.798778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:41.798778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:20:41.798778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:20:41.798778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:20:41.798778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:20:41.799766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:20:41.799766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:20:41.799766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:20:41.799766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:20:41.799766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:41.799766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:41.799766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:41.799766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:41.799766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:41.799766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:41.799766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:41.799766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:41.799766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:41.799766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:41.799766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:41.799766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:41.800606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:41.800606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:41.800606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:41.800606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:41.800606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:20:41.800606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:20:41.800606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:20:41.800606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:20:41.800606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:20:41.800606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:20:41.800606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 10:20:41.800606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:41.800606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 492/965] 10.222.0.49 () {56 vars in 1120 bytes} [Mon Sep 26 10:20:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:41.805279 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55117746-155d-4cb7-84d7-076624b2d623 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:41.824908 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 474/966] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Sep 26 10:20:41 2022] POST /identity/v3/services => generated 242 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:20:41.933921 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55117746-155d-4cb7-84d7-076624b2d623 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:41.981822 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03001b7a-f800-4829-8ae8-82e8d09468d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:41.982475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03001b7a-f800-4829-8ae8-82e8d09468d9 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:41.982750 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03001b7a-f800-4829-8ae8-82e8d09468d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:41.983018 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03001b7a-f800-4829-8ae8-82e8d09468d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:41.983759 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03001b7a-f800-4829-8ae8-82e8d09468d9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:41.984062 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03001b7a-f800-4829-8ae8-82e8d09468d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:41.985454 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03001b7a-f800-4829-8ae8-82e8d09468d9 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:41.995286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 493/967] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Sep 26 10:20:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:42.001844 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e5b3a2-0ee2-4f92-a04f-6072175c3482 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:42.065449 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:42.127020 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e5b3a2-0ee2-4f92-a04f-6072175c3482 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:42.163991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7badd-c43c-46e9-9333-71041f24d22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:42.164646 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7badd-c43c-46e9-9333-71041f24d22b None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:42.164924 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7badd-c43c-46e9-9333-71041f24d22b None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:42.165194 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7badd-c43c-46e9-9333-71041f24d22b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:42.178398 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7badd-c43c-46e9-9333-71041f24d22b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:42.179867 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7badd-c43c-46e9-9333-71041f24d22b None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:42.196373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-41b7badd-c43c-46e9-9333-71041f24d22b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:42.196373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:42.196373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:42.196373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:42.196373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:42.196373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:42.196373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:42.196373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:42.196373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:42.196373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:42.196373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:42.196373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:42.196373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:42.196373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:42.196373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:42.196373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:42.197343 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:42.197343 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:42.197343 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:42.197343 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:42.197343 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:42.197343 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:42.197343 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:42.197343 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:42.197343 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:42.197343 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:42.197343 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:42.197343 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:42.197343 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:42.197343 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:42.197343 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:42.197343 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:42.197343 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:42.198273 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:42.198273 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:42.198273 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:42.198273 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 475/968] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:42.203150 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03001b7a-f800-4829-8ae8-82e8d09468d9 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:42.233138 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:42.286274 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f6f6f-c780-4069-8ec3-edd54c871f66 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:42.286865 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f6f6f-c780-4069-8ec3-edd54c871f66 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:42.286865 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f6f6f-c780-4069-8ec3-edd54c871f66 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:42.288489 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 476/969] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:42.294899 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38818890-2acc-4090-bb57-ba0cda96c4aa None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:42.295177 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38818890-2acc-4090-bb57-ba0cda96c4aa None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:42.295558 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38818890-2acc-4090-bb57-ba0cda96c4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:42.315479 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-38818890-2acc-4090-bb57-ba0cda96c4aa None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:42.316789 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-38818890-2acc-4090-bb57-ba0cda96c4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:42.321175 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03001b7a-f800-4829-8ae8-82e8d09468d9 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:42.361302 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab228c-c2e9-42fa-80b0-c737b7ab7911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:42.362181 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab228c-c2e9-42fa-80b0-c737b7ab7911 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:42.362601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab228c-c2e9-42fa-80b0-c737b7ab7911 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:42.363036 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab228c-c2e9-42fa-80b0-c737b7ab7911 None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:42.363880 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbab228c-c2e9-42fa-80b0-c737b7ab7911 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:42.364337 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbab228c-c2e9-42fa-80b0-c737b7ab7911 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:42.365952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbab228c-c2e9-42fa-80b0-c737b7ab7911 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:42.384300 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 494/970] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:42.390804 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab228c-c2e9-42fa-80b0-c737b7ab7911 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:42.402261 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 477/971] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:42 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 10:20:42.404645 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8974272-819a-4db8-9a90-da48afdf549a None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:42.405003 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8974272-819a-4db8-9a90-da48afdf549a None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:42.405352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8974272-819a-4db8-9a90-da48afdf549a None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:42.407222 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 478/972] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:42.412577 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2530d-1cf1-4939-af07-b2f6161ff820 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:42.412864 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2530d-1cf1-4939-af07-b2f6161ff820 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:42.413142 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2530d-1cf1-4939-af07-b2f6161ff820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:42.433045 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-31f2530d-1cf1-4939-af07-b2f6161ff820 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:42.434287 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-31f2530d-1cf1-4939-af07-b2f6161ff820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:42.459885 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:42.506756 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:42.519439 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 479/973] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:42 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 10:20:42.527405 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab228c-c2e9-42fa-80b0-c737b7ab7911 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:42.564881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aac486-0b76-46ed-b154-4cc986efa976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:42.565865 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aac486-0b76-46ed-b154-4cc986efa976 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:42.566381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aac486-0b76-46ed-b154-4cc986efa976 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:42.566945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aac486-0b76-46ed-b154-4cc986efa976 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:42.574521 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aac486-0b76-46ed-b154-4cc986efa976 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:42.576203 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aac486-0b76-46ed-b154-4cc986efa976 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:42.588530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-53aac486-0b76-46ed-b154-4cc986efa976 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 10:20:42.588530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:42.588530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:42.588530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:42.588530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:42.588530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:42.588530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:42.588530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:42.588530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:42.588530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:42.588530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:42.588530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:42.588530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:20:42.588530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:20:42.588530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:20:42.588530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:20:42.589493 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:42.589493 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:42.589493 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:20:42.589493 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:42.589493 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:20:42.589493 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:42.589493 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:42.589493 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:42.589493 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:42.589493 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:42.589493 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:42.589493 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:42.589493 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:42.589493 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:42.589493 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:42.589493 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:42.589493 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:42.589493 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:42.590469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:42.590469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:42.590469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:20:42.590469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:20:42.590469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:20:42.590469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:20:42.590469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:20:42.590469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:20:42.590469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 10:20:42.590469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:42.593487 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 495/974] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Sep 26 10:20:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:20:42.602266 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2530d-1cf1-4939-af07-b2f6161ff820 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:42.674068 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aac486-0b76-46ed-b154-4cc986efa976 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:42.717681 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2530d-1cf1-4939-af07-b2f6161ff820 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:42.764059 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656f458-3be8-4aa2-aedf-49250fa1527b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:42.765220 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656f458-3be8-4aa2-aedf-49250fa1527b None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:42.765741 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656f458-3be8-4aa2-aedf-49250fa1527b None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:42.766047 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656f458-3be8-4aa2-aedf-49250fa1527b None admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:42.766781 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0656f458-3be8-4aa2-aedf-49250fa1527b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:42.767329 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0656f458-3be8-4aa2-aedf-49250fa1527b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:42.769135 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0656f458-3be8-4aa2-aedf-49250fa1527b None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:42.791154 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aac486-0b76-46ed-b154-4cc986efa976 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:42.802121 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 480/975] 10.222.0.49 () {56 vars in 1149 bytes} [Mon Sep 26 10:20:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:42.806516 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656f458-3be8-4aa2-aedf-49250fa1527b None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:42.828919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:42.839326 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb1049-7c18-4c4b-8c74-01fcd2f31baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:42.839951 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb1049-7c18-4c4b-8c74-01fcd2f31baa None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:42.840229 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb1049-7c18-4c4b-8c74-01fcd2f31baa None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:42.840495 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb1049-7c18-4c4b-8c74-01fcd2f31baa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:42.841108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb1049-7c18-4c4b-8c74-01fcd2f31baa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:42.842483 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb1049-7c18-4c4b-8c74-01fcd2f31baa None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:42.852412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-92cb1049-7c18-4c4b-8c74-01fcd2f31baa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 10:20:42.852412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:42.852412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 10:20:42.852412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 10:20:42.852412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 10:20:42.852412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 10:20:42.852412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 10:20:42.852412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:42.852412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 10:20:42.852412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:42.852412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 10:20:42.852412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:42.852412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:42.852412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:42.852412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:42.852412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:42.852412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:42.853457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:42.853457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:42.853457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:42.853457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:42.853457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:42.853457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:42.853457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 10:20:42.853457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 10:20:42.853457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 10:20:42.853457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 10:20:42.853457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:42.853457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:42.853457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:42.853457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:42.853457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:42.853457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:42.854383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:42.854383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:42.854383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:42.854383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:42.854383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:42.854383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:42.854383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:42.854383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:42.854383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:42.854383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:42.854383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 10:20:42.854383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:42.854383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 10:20:42.854383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:42.854383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 496/976] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Sep 26 10:20:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:42.855241 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb1049-7c18-4c4b-8c74-01fcd2f31baa None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:42.867641 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:42.925423 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656f458-3be8-4aa2-aedf-49250fa1527b None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:42.962620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eec3e3-dabf-40c5-8209-8f98cc84ff7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:42.963221 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eec3e3-dabf-40c5-8209-8f98cc84ff7d None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:42.963503 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eec3e3-dabf-40c5-8209-8f98cc84ff7d None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:42.963777 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eec3e3-dabf-40c5-8209-8f98cc84ff7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:42.964458 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb1049-7c18-4c4b-8c74-01fcd2f31baa None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:43.000180 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3209ae98-5b75-4507-b182-b21f73d41775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:43.000708 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209ae98-5b75-4507-b182-b21f73d41775 None admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:43.000984 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209ae98-5b75-4507-b182-b21f73d41775 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:43.001254 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209ae98-5b75-4507-b182-b21f73d41775 None admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:43.001884 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3209ae98-5b75-4507-b182-b21f73d41775 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:43.003321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3209ae98-5b75-4507-b182-b21f73d41775 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:43.018764 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.sql.core [None req-3209ae98-5b75-4507-b182-b21f73d41775 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 26 10:20:43.018764 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 26 10:20:43.018764 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [parameters: {'user_id': 'fa62fe7baf604df8b4bc16b4aec5d664', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 26 10:20:43.018764 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81420) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 26 10:20:43.025996 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-3209ae98-5b75-4507-b182-b21f73d41775 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 26 10:20:43.025996 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:43.025996 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 26 10:20:43.025996 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 26 10:20:43.025996 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 26 10:20:43.025996 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 26 10:20:43.025996 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 26 10:20:43.025996 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application result = self._query(query) Sep 26 10:20:43.025996 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 26 10:20:43.025996 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application conn.query(q) Sep 26 10:20:43.025996 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 26 10:20:43.025996 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 26 10:20:43.025996 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 26 10:20:43.025996 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application result.read() Sep 26 10:20:43.025996 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 26 10:20:43.027090 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 26 10:20:43.027090 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 26 10:20:43.027090 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 26 10:20:43.027090 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 26 10:20:43.027090 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 26 10:20:43.027090 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 26 10:20:43.027090 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 26 10:20:43.027090 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 26 10:20:43.027090 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:43.027090 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 26 10:20:43.027090 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:43.027090 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:43.027090 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 26 10:20:43.027090 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 26 10:20:43.027090 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 26 10:20:43.027090 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 26 10:20:43.027090 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 26 10:20:43.027090 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application next(self.gen) Sep 26 10:20:43.028075 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 26 10:20:43.028075 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application yield resource Sep 26 10:20:43.028075 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 26 10:20:43.028075 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application next(self.gen) Sep 26 10:20:43.028075 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 26 10:20:43.028075 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self.session.rollback() Sep 26 10:20:43.028075 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 26 10:20:43.028075 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self.force_reraise() Sep 26 10:20:43.028075 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 26 10:20:43.028075 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise self.value Sep 26 10:20:43.028075 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 26 10:20:43.028075 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 26 10:20:43.028075 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 26 10:20:43.028075 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application session.commit() Sep 26 10:20:43.028075 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 26 10:20:43.028075 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 26 10:20:43.028075 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 26 10:20:43.029026 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._prepare_impl() Sep 26 10:20:43.029026 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 26 10:20:43.029026 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self.session.flush() Sep 26 10:20:43.029026 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 26 10:20:43.029026 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._flush(objects) Sep 26 10:20:43.029026 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 26 10:20:43.029026 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 26 10:20:43.029026 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 26 10:20:43.029026 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application compat.raise_( Sep 26 10:20:43.029026 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 26 10:20:43.029026 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception Sep 26 10:20:43.029026 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 26 10:20:43.029026 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application flush_context.execute() Sep 26 10:20:43.029026 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 26 10:20:43.029026 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rec.execute(self) Sep 26 10:20:43.029026 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 26 10:20:43.029026 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 26 10:20:43.029915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 26 10:20:43.029915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 26 10:20:43.029915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 26 10:20:43.029915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 26 10:20:43.029915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 26 10:20:43.029915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 26 10:20:43.029915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 26 10:20:43.029915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 26 10:20:43.029915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 26 10:20:43.029915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 26 10:20:43.029915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 26 10:20:43.029915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 26 10:20:43.029915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 26 10:20:43.029915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 26 10:20:43.029915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 26 10:20:43.029915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception Sep 26 10:20:43.030809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 26 10:20:43.030809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 26 10:20:43.030809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 26 10:20:43.030809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 26 10:20:43.030809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 26 10:20:43.030809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application result = self._query(query) Sep 26 10:20:43.030809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 26 10:20:43.030809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application conn.query(q) Sep 26 10:20:43.030809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 26 10:20:43.030809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 26 10:20:43.030809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 26 10:20:43.030809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application result.read() Sep 26 10:20:43.030809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 26 10:20:43.030809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 26 10:20:43.030809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 26 10:20:43.030809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 26 10:20:43.031673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 26 10:20:43.031673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 26 10:20:43.031673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 26 10:20:43.031673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 26 10:20:43.031673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 26 10:20:43.031673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 26 10:20:43.031673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [parameters: {'user_id': 'fa62fe7baf604df8b4bc16b4aec5d664', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 26 10:20:43.031673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 26 10:20:43.031673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:43.031673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 10:20:43.031673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:43.031673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:43.031673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:43.031673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:43.031673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:43.032482 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:43.032482 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:43.032482 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:43.032482 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:43.032482 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:43.032482 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:43.032482 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:43.032482 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 26 10:20:43.032482 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 26 10:20:43.032482 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:43.032482 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:43.032482 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:20:43.032482 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:43.032482 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:20:43.032482 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:43.032482 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 26 10:20:43.032482 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 26 10:20:43.033513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 26 10:20:43.033513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 26 10:20:43.033513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 26 10:20:43.033513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 26 10:20:43.033513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 26 10:20:43.033513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:43.033513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 497/977] 10.222.0.49 () {60 vars in 1131 bytes} [Mon Sep 26 10:20:42 2022] POST /identity/v3/users => generated 160 bytes in 173 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:43.033513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3209ae98-5b75-4507-b182-b21f73d41775 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:43.043800 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eec3e3-dabf-40c5-8209-8f98cc84ff7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:43.046218 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eec3e3-dabf-40c5-8209-8f98cc84ff7d None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:43.064066 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:43.112087 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:43.145954 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 481/978] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:20:42 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 10:20:43.146824 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eec3e3-dabf-40c5-8209-8f98cc84ff7d None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:43.156044 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3209ae98-5b75-4507-b182-b21f73d41775 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:43.194316 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45fc3e-d85a-44b9-b911-9d2a41409d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:43.195013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45fc3e-d85a-44b9-b911-9d2a41409d4a None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:43.195354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45fc3e-d85a-44b9-b911-9d2a41409d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:43.195687 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45fc3e-d85a-44b9-b911-9d2a41409d4a None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:43.196380 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45fc3e-d85a-44b9-b911-9d2a41409d4a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:43.196851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45fc3e-d85a-44b9-b911-9d2a41409d4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:43.198656 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45fc3e-d85a-44b9-b911-9d2a41409d4a None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:43.210190 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 498/979] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Sep 26 10:20:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:43.210190 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45fc3e-d85a-44b9-b911-9d2a41409d4a None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:43.257549 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eec3e3-dabf-40c5-8209-8f98cc84ff7d None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:43.275933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:43.297022 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf20031-40dc-4d87-8ce8-35b7096c281e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:43.297661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf20031-40dc-4d87-8ce8-35b7096c281e None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:43.297943 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf20031-40dc-4d87-8ce8-35b7096c281e None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:43.298242 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf20031-40dc-4d87-8ce8-35b7096c281e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:43.305446 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf20031-40dc-4d87-8ce8-35b7096c281e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:43.306909 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf20031-40dc-4d87-8ce8-35b7096c281e None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:43.318368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45fc3e-d85a-44b9-b911-9d2a41409d4a None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:43.319265 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-acf20031-40dc-4d87-8ce8-35b7096c281e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 10:20:43.319265 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:43.319265 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:43.319265 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:43.319265 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:43.319265 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:43.319265 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:43.319265 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:43.319265 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:43.319265 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:43.319265 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:43.319265 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:43.319265 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:20:43.319265 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:20:43.319265 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:20:43.319265 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:20:43.320206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:43.320206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:43.320206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:20:43.320206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:43.320206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:20:43.320206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:43.320206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:43.320206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:43.320206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:43.320206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:43.320206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:43.320206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:43.320206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:43.320206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:43.320206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:43.320206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:43.320206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:43.320206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:43.321139 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:43.321139 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:43.321139 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:20:43.321139 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:20:43.321139 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:20:43.321139 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:20:43.321139 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:20:43.321139 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:20:43.321139 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 10:20:43.321139 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:43.321139 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 482/980] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:20:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:43.321139 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf20031-40dc-4d87-8ce8-35b7096c281e None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:43.360459 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d841d5-8989-4d2e-8ee5-4b92fee78c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:43.361113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d841d5-8989-4d2e-8ee5-4b92fee78c4e None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:43.361394 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d841d5-8989-4d2e-8ee5-4b92fee78c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:43.361665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d841d5-8989-4d2e-8ee5-4b92fee78c4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:43.363552 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:43.372845 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d841d5-8989-4d2e-8ee5-4b92fee78c4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:43.374362 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d841d5-8989-4d2e-8ee5-4b92fee78c4e None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:43.391359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-69d841d5-8989-4d2e-8ee5-4b92fee78c4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:43.391359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:43.391359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:43.391359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:43.391359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:43.391359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:43.391359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:43.391359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:43.391359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:43.391359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:43.391359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:43.391359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:43.391359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:43.391359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:43.391359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:43.391359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:43.392354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:43.392354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:43.392354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:43.392354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:43.392354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:43.392354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:43.392354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:43.392354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:43.392354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:43.392354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:43.392354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:43.392354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:43.392354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:43.392354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:43.392354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:43.392354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:43.392354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:43.393273 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:43.393273 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:43.393273 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:43.398461 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 499/981] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:43.398461 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d841d5-8989-4d2e-8ee5-4b92fee78c4e None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:43.434392 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf20031-40dc-4d87-8ce8-35b7096c281e None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:43.471658 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b337b8-a707-4cfc-8cf6-e0b01bc4c4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:43.472583 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b337b8-a707-4cfc-8cf6-e0b01bc4c4c9 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:43.473036 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b337b8-a707-4cfc-8cf6-e0b01bc4c4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:43.473490 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b337b8-a707-4cfc-8cf6-e0b01bc4c4c9 None admin] PATH_INFO: `/v3/services` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:43.474149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b337b8-a707-4cfc-8cf6-e0b01bc4c4c9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:43.474446 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b337b8-a707-4cfc-8cf6-e0b01bc4c4c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:43.475799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b337b8-a707-4cfc-8cf6-e0b01bc4c4c9 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:43.485830 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 483/982] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Sep 26 10:20:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:43.485830 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b337b8-a707-4cfc-8cf6-e0b01bc4c4c9 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:43.508431 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d841d5-8989-4d2e-8ee5-4b92fee78c4e None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:43.531910 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:43.545317 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c41c62-3497-4fa5-bd82-6fabe455a6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:43.545948 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c41c62-3497-4fa5-bd82-6fabe455a6c3 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:43.546252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c41c62-3497-4fa5-bd82-6fabe455a6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:43.546523 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c41c62-3497-4fa5-bd82-6fabe455a6c3 None admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:43.547221 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c41c62-3497-4fa5-bd82-6fabe455a6c3 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:43.547519 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c41c62-3497-4fa5-bd82-6fabe455a6c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:43.549013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c41c62-3497-4fa5-bd82-6fabe455a6c3 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:43.584664 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 500/983] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Sep 26 10:20:43 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:43.593799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b337b8-a707-4cfc-8cf6-e0b01bc4c4c9 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:43.595160 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c41c62-3497-4fa5-bd82-6fabe455a6c3 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:43.633104 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ea22a-b934-430a-8265-9912586532de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:43.633772 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ea22a-b934-430a-8265-9912586532de None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:43.634124 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ea22a-b934-430a-8265-9912586532de None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:43.634396 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ea22a-b934-430a-8265-9912586532de None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:43.635130 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ea22a-b934-430a-8265-9912586532de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:43.635392 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ea22a-b934-430a-8265-9912586532de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:43.636882 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ea22a-b934-430a-8265-9912586532de None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:43.656739 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 484/984] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:43.657716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ea22a-b934-430a-8265-9912586532de None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:43.667494 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:43.749861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c41c62-3497-4fa5-bd82-6fabe455a6c3 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:43.780692 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ea22a-b934-430a-8265-9912586532de None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:43.782288 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:43.806181 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeceb02-f86b-459a-a908-a29efea5d41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:43.806807 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeceb02-f86b-459a-a908-a29efea5d41d None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:43.807087 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeceb02-f86b-459a-a908-a29efea5d41d None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:43.807362 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeceb02-f86b-459a-a908-a29efea5d41d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:43.808086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeceb02-f86b-459a-a908-a29efea5d41d None admin] RBAC: Adding query filter params (interface=admin, service_id=2e7c9b695cab465c8127d3715e5cc9fe, region_id=RegionOne) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:43.808394 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeceb02-f86b-459a-a908-a29efea5d41d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:43.811098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeceb02-f86b-459a-a908-a29efea5d41d None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:43.819491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 501/985] 10.222.0.49 () {56 vars in 1249 bytes} [Mon Sep 26 10:20:43 2022] GET /identity/v3/endpoints?service_id=2e7c9b695cab465c8127d3715e5cc9fe&interface=admin®ion_id=RegionOne => generated 193 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:43.825848 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeceb02-f86b-459a-a908-a29efea5d41d None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:43.835466 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e9ee1-b291-42cb-8f37-16faa99a8102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:43.837309 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e9ee1-b291-42cb-8f37-16faa99a8102 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:43.837853 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e9ee1-b291-42cb-8f37-16faa99a8102 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:43.838395 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e9ee1-b291-42cb-8f37-16faa99a8102 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:43.846598 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e9ee1-b291-42cb-8f37-16faa99a8102 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:43.848872 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e9ee1-b291-42cb-8f37-16faa99a8102 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:43.861985 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-067e9ee1-b291-42cb-8f37-16faa99a8102 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:43.861985 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:43.861985 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:43.861985 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:43.861985 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:43.861985 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:43.861985 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:43.861985 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:43.861985 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:43.861985 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:43.861985 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:43.861985 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:43.861985 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:20:43.861985 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:20:43.861985 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:20:43.862931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:20:43.862931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:43.862931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:43.862931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:43.862931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:43.862931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:43.862931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:43.862931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:43.862931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:43.862931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:43.862931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:43.862931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:43.862931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:43.862931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:43.862931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:43.862931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:43.862931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:43.863964 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:20:43.863964 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:20:43.863964 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:20:43.863964 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:20:43.863964 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:20:43.863964 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:20:43.863964 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:43.863964 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:43.866012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 485/986] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:20:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:43.873473 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e9ee1-b291-42cb-8f37-16faa99a8102 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:43.952667 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeceb02-f86b-459a-a908-a29efea5d41d None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:43.964767 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:43.995736 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e9ee1-b291-42cb-8f37-16faa99a8102 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:43.996681 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a647c-823c-4723-b626-ffb23822a03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:43.997351 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a647c-823c-4723-b626-ffb23822a03f None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:43.997631 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a647c-823c-4723-b626-ffb23822a03f None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:43.997905 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a647c-823c-4723-b626-ffb23822a03f None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:43.998570 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a647c-823c-4723-b626-ffb23822a03f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:43.999947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a647c-823c-4723-b626-ffb23822a03f None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:44.014063 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 502/987] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Sep 26 10:20:43 2022] GET /identity/v3/services => generated 1727 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:20:44.042522 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e73eda-5ab1-4a33-977d-29a05c52321a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:44.043053 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e73eda-5ab1-4a33-977d-29a05c52321a None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:44.043326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e73eda-5ab1-4a33-977d-29a05c52321a None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:44.043593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e73eda-5ab1-4a33-977d-29a05c52321a None admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:44.044884 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e73eda-5ab1-4a33-977d-29a05c52321a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:44.045185 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e73eda-5ab1-4a33-977d-29a05c52321a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:44.046667 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e73eda-5ab1-4a33-977d-29a05c52321a None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:44.060199 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 486/988] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:20:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:44.066957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a647c-823c-4723-b626-ffb23822a03f None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:44.087731 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83da274-7f0c-4ad3-a2ea-bd666e508402 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:44.088677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83da274-7f0c-4ad3-a2ea-bd666e508402 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:44.089622 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83da274-7f0c-4ad3-a2ea-bd666e508402 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:44.092206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 487/989] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:20:44.097534 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c25413-5a8a-44ad-b56f-e2404f994faa None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:44.097819 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c25413-5a8a-44ad-b56f-e2404f994faa None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:44.098204 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c25413-5a8a-44ad-b56f-e2404f994faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:44.121980 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-16c25413-5a8a-44ad-b56f-e2404f994faa None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:44.123806 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-16c25413-5a8a-44ad-b56f-e2404f994faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:44.144611 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:44.193150 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a647c-823c-4723-b626-ffb23822a03f None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:44.205586 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 488/990] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:44 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 10:20:44.230033 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312316e2-1617-451f-8536-f92245dbf252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:44.230646 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312316e2-1617-451f-8536-f92245dbf252 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:44.230924 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312316e2-1617-451f-8536-f92245dbf252 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:44.231194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312316e2-1617-451f-8536-f92245dbf252 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:44.231943 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312316e2-1617-451f-8536-f92245dbf252 None admin] RBAC: Adding query filter params (role.id=dabb7e453c8742aa809ef7543c15d1d3, scope.project.id=cff3a6011f4c4893bddffdad3d69f3df, user.id=9158557f1a2947e2815dc8d586d8b0c3) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:44.232247 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312316e2-1617-451f-8536-f92245dbf252 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:44.233703 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312316e2-1617-451f-8536-f92245dbf252 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:44.242059 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 503/991] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:20:44 2022] GET /identity/v3/role_assignments?user.id=9158557f1a2947e2815dc8d586d8b0c3&scope.project.id=cff3a6011f4c4893bddffdad3d69f3df&role.id=dabb7e453c8742aa809ef7543c15d1d3 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:44.486708 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c25413-5a8a-44ad-b56f-e2404f994faa None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:44.588138 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c25413-5a8a-44ad-b56f-e2404f994faa None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:44.622935 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236fcbf0-a4cb-41e7-aa75-b539b3d401f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:44.623714 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236fcbf0-a4cb-41e7-aa75-b539b3d401f1 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:44.624092 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236fcbf0-a4cb-41e7-aa75-b539b3d401f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:44.624462 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236fcbf0-a4cb-41e7-aa75-b539b3d401f1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:44.625138 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236fcbf0-a4cb-41e7-aa75-b539b3d401f1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:44.626633 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236fcbf0-a4cb-41e7-aa75-b539b3d401f1 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:44.635448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-236fcbf0-a4cb-41e7-aa75-b539b3d401f1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 10:20:44.635448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:44.635448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 10:20:44.635448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 10:20:44.635448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 10:20:44.635448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 10:20:44.635448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 10:20:44.635448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:44.635448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 10:20:44.635448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:44.635448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 10:20:44.635448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:44.635448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:44.635448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:44.635448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:44.635448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:44.635448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:44.636632 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:44.636632 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:44.636632 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:44.636632 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:44.636632 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:44.636632 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:44.636632 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 10:20:44.636632 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 10:20:44.636632 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 10:20:44.636632 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 10:20:44.636632 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:44.636632 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:44.636632 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:44.636632 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:44.636632 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:44.636632 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:44.637512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:44.637512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:44.637512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:44.637512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:44.637512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:44.637512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:44.637512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:44.637512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:44.637512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:44.637512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:44.637512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 10:20:44.637512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:44.637512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 10:20:44.637512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:44.639491 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 489/992] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:20:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:44.643796 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312316e2-1617-451f-8536-f92245dbf252 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:44.678463 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:44.745766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312316e2-1617-451f-8536-f92245dbf252 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:44.782646 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c7133-58c0-4b69-93fe-d0f3f56cf7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:44.783253 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c7133-58c0-4b69-93fe-d0f3f56cf7a3 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:44.783532 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c7133-58c0-4b69-93fe-d0f3f56cf7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:44.783803 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c7133-58c0-4b69-93fe-d0f3f56cf7a3 None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:44.784426 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c7133-58c0-4b69-93fe-d0f3f56cf7a3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:44.784727 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c7133-58c0-4b69-93fe-d0f3f56cf7a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:44.786103 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c7133-58c0-4b69-93fe-d0f3f56cf7a3 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:44.795768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 504/993] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Sep 26 10:20:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:44.797127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236fcbf0-a4cb-41e7-aa75-b539b3d401f1 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:44.820339 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:44.906188 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236fcbf0-a4cb-41e7-aa75-b539b3d401f1 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:44.942131 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce3081-81ae-405c-a64e-b8d036d415f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:44.942738 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce3081-81ae-405c-a64e-b8d036d415f3 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:44.943018 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce3081-81ae-405c-a64e-b8d036d415f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:44.943287 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce3081-81ae-405c-a64e-b8d036d415f3 None admin] PATH_INFO: `/v3/services` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:44.943914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce3081-81ae-405c-a64e-b8d036d415f3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:44.944216 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce3081-81ae-405c-a64e-b8d036d415f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:44.951810 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce3081-81ae-405c-a64e-b8d036d415f3 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:44.959087 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 490/994] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Sep 26 10:20:44 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:44.964889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c7133-58c0-4b69-93fe-d0f3f56cf7a3 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:45.063535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:45.067271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c7133-58c0-4b69-93fe-d0f3f56cf7a3 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:45.100735 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd951a-0de3-426d-a5a2-4e01ff795c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:45.101338 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd951a-0de3-426d-a5a2-4e01ff795c84 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:45.101619 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd951a-0de3-426d-a5a2-4e01ff795c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:45.101893 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd951a-0de3-426d-a5a2-4e01ff795c84 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:45.102686 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd951a-0de3-426d-a5a2-4e01ff795c84 None admin] RBAC: Adding query filter params (interface=public, service_id=a912b7de87be477a999eb6e8240a38b4, region_id=RegionOne) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:45.102993 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd951a-0de3-426d-a5a2-4e01ff795c84 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:45.104359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd951a-0de3-426d-a5a2-4e01ff795c84 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:45.109971 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 505/995] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Sep 26 10:20:44 2022] GET /identity/v3/endpoints?service_id=a912b7de87be477a999eb6e8240a38b4&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:45.115611 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce3081-81ae-405c-a64e-b8d036d415f3 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:45.199488 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:45.219693 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce3081-81ae-405c-a64e-b8d036d415f3 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:45.255487 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5422f434-1712-4564-9214-13b4fa51c51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:45.256091 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422f434-1712-4564-9214-13b4fa51c51d None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:45.256368 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422f434-1712-4564-9214-13b4fa51c51d None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:45.256639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422f434-1712-4564-9214-13b4fa51c51d None admin] PATH_INFO: `/v3/services` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:45.257237 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5422f434-1712-4564-9214-13b4fa51c51d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:45.258636 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5422f434-1712-4564-9214-13b4fa51c51d None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:45.270123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 491/996] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Sep 26 10:20:45 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:20:46.213147 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2a927-2298-42bd-9a65-24b7509f8bb2 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:46.213657 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2a927-2298-42bd-9a65-24b7509f8bb2 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:46.213657 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2a927-2298-42bd-9a65-24b7509f8bb2 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:46.215125 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 506/997] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:46.220663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b55a68-b1ee-458a-a5cf-095f51167bfe None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:46.221066 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b55a68-b1ee-458a-a5cf-095f51167bfe None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:46.221228 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b55a68-b1ee-458a-a5cf-095f51167bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:46.240008 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-11b55a68-b1ee-458a-a5cf-095f51167bfe None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:46.241207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-11b55a68-b1ee-458a-a5cf-095f51167bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:46.311616 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 492/998] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:46 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:20:46.413914 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813330f-f708-48a3-9dbc-bf999edff395 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:46.414332 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813330f-f708-48a3-9dbc-bf999edff395 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:46.414514 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813330f-f708-48a3-9dbc-bf999edff395 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:46.415961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 507/999] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:46.420999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6821d-c800-4371-a018-d9d0dcebfc59 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:46.421282 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6821d-c800-4371-a018-d9d0dcebfc59 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:46.421553 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6821d-c800-4371-a018-d9d0dcebfc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:46.440499 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-9cb6821d-c800-4371-a018-d9d0dcebfc59 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:46.441674 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-9cb6821d-c800-4371-a018-d9d0dcebfc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:46.508376 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 493/1000] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:46 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:20:46.575994 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7813330f-f708-48a3-9dbc-bf999edff395 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:46.681344 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7813330f-f708-48a3-9dbc-bf999edff395 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:46.715010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf50c144-276e-400e-9050-2b6220ae122c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:46.715638 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf50c144-276e-400e-9050-2b6220ae122c None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:46.715944 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf50c144-276e-400e-9050-2b6220ae122c None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:46.716219 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf50c144-276e-400e-9050-2b6220ae122c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:46.716798 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf50c144-276e-400e-9050-2b6220ae122c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:46.718189 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf50c144-276e-400e-9050-2b6220ae122c None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:46.726829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-bf50c144-276e-400e-9050-2b6220ae122c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 10:20:46.726829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:46.726829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 10:20:46.726829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 10:20:46.726829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 10:20:46.726829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 10:20:46.726829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 10:20:46.726829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:46.726829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 10:20:46.726829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:46.726829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 10:20:46.726829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:46.726829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:46.726829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:46.726829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:46.726829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:46.726829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:46.727846 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:46.727846 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:46.727846 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:46.727846 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:46.727846 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:46.727846 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:46.727846 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 10:20:46.727846 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 10:20:46.727846 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 10:20:46.727846 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 10:20:46.727846 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:46.727846 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:46.727846 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:46.727846 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:46.727846 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:46.727846 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:46.728695 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:46.728695 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:46.728695 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:46.728695 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:46.728695 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:46.728695 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:46.728695 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:46.728695 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:46.728695 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:46.728695 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:46.728695 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 10:20:46.728695 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:46.728695 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 10:20:46.728695 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:46.728695 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 508/1001] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Sep 26 10:20:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:46.736043 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6821d-c800-4371-a018-d9d0dcebfc59 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:46.740510 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:46.773346 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf50c144-276e-400e-9050-2b6220ae122c None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:46.803360 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:46.851175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6821d-c800-4371-a018-d9d0dcebfc59 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:46.883675 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf50c144-276e-400e-9050-2b6220ae122c None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:46.889463 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa64ee-517d-4e0c-8420-fa26bd9b7374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:46.890076 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa64ee-517d-4e0c-8420-fa26bd9b7374 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:46.890350 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa64ee-517d-4e0c-8420-fa26bd9b7374 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:46.890942 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa64ee-517d-4e0c-8420-fa26bd9b7374 None admin] PATH_INFO: `/v3/services` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:46.891227 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa64ee-517d-4e0c-8420-fa26bd9b7374 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:46.891518 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa64ee-517d-4e0c-8420-fa26bd9b7374 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:46.893004 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa64ee-517d-4e0c-8420-fa26bd9b7374 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:46.897639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 494/1002] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Sep 26 10:20:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:46.903216 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa64ee-517d-4e0c-8420-fa26bd9b7374 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:46.928173 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ef2c5-9a83-4272-9de3-0554776633a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:46.928928 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ef2c5-9a83-4272-9de3-0554776633a2 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:46.929416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ef2c5-9a83-4272-9de3-0554776633a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:46.929534 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ef2c5-9a83-4272-9de3-0554776633a2 None admin] PATH_INFO: `/v3/services/image` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:46.930085 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219ef2c5-9a83-4272-9de3-0554776633a2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:46.931420 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219ef2c5-9a83-4272-9de3-0554776633a2 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:46.940291 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-219ef2c5-9a83-4272-9de3-0554776633a2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 10:20:46.940291 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:46.940291 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 10:20:46.940291 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 10:20:46.940291 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 10:20:46.940291 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 10:20:46.940291 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 10:20:46.940291 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:46.940291 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 10:20:46.940291 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:46.940291 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 10:20:46.940291 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:46.940291 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:46.940291 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:46.940291 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:46.940291 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:46.940291 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:46.941310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:46.941310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:46.941310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:46.941310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:46.941310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:46.941310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:46.941310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 10:20:46.941310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 10:20:46.941310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 10:20:46.941310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 10:20:46.941310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:46.941310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:46.941310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:46.941310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:46.941310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:46.941310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:46.942243 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:46.942243 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:46.942243 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:46.942243 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:46.942243 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:46.942243 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:46.942243 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:46.942243 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:46.942243 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:46.942243 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:46.942243 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 10:20:46.942243 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:46.942243 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 10:20:46.942243 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:46.942243 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 509/1003] 10.222.0.49 () {56 vars in 1105 bytes} [Mon Sep 26 10:20:46 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:46.949243 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ef2c5-9a83-4272-9de3-0554776633a2 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:46.973854 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:47.010642 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa64ee-517d-4e0c-8420-fa26bd9b7374 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:47.048564 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc38718-6949-4837-948b-72416a6c457d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:47.049109 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc38718-6949-4837-948b-72416a6c457d None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:47.049388 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc38718-6949-4837-948b-72416a6c457d None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:47.049654 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc38718-6949-4837-948b-72416a6c457d None admin] PATH_INFO: `/v3/services` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:47.049928 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:47.050323 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc38718-6949-4837-948b-72416a6c457d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:47.050622 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc38718-6949-4837-948b-72416a6c457d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:47.052046 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc38718-6949-4837-948b-72416a6c457d None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:47.060127 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ef2c5-9a83-4272-9de3-0554776633a2 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:47.062180 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 495/1004] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Sep 26 10:20:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:47.064301 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc38718-6949-4837-948b-72416a6c457d None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:47.099603 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d36262-e97f-445b-bb82-b2743895a16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:47.100209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d36262-e97f-445b-bb82-b2743895a16d None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:47.100505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d36262-e97f-445b-bb82-b2743895a16d None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:47.100780 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d36262-e97f-445b-bb82-b2743895a16d None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:47.101383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d36262-e97f-445b-bb82-b2743895a16d None admin] RBAC: Adding query filter params (name=image) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:47.101713 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d36262-e97f-445b-bb82-b2743895a16d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:47.103089 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d36262-e97f-445b-bb82-b2743895a16d None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:47.108949 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 510/1005] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:47.114440 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d36262-e97f-445b-bb82-b2743895a16d None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:47.164682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc38718-6949-4837-948b-72416a6c457d None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:47.193436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:47.199009 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707ede8-821b-404c-85e9-70e9f1132028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:47.199621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707ede8-821b-404c-85e9-70e9f1132028 None admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:47.199892 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707ede8-821b-404c-85e9-70e9f1132028 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:47.200180 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707ede8-821b-404c-85e9-70e9f1132028 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:47.200840 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4707ede8-821b-404c-85e9-70e9f1132028 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:47.202214 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4707ede8-821b-404c-85e9-70e9f1132028 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:47.226082 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 496/1006] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:20:47 2022] POST /identity/v3/endpoints => generated 325 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:20:47.237943 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d36262-e97f-445b-bb82-b2743895a16d None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:47.283754 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c95fa5-483d-4e90-8933-7082905826af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:47.284374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c95fa5-483d-4e90-8933-7082905826af None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:47.284654 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c95fa5-483d-4e90-8933-7082905826af None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:47.284927 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c95fa5-483d-4e90-8933-7082905826af None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:47.285569 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c95fa5-483d-4e90-8933-7082905826af None admin] RBAC: Adding query filter params (type=image) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:47.285875 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c95fa5-483d-4e90-8933-7082905826af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:47.287287 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c95fa5-483d-4e90-8933-7082905826af None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:47.292418 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 511/1007] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:47 2022] GET /identity/v3/services?type=image => generated 122 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:47.449966 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b99128-6ab5-4d91-b9cb-2a05dbcf4637 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:47.450782 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b99128-6ab5-4d91-b9cb-2a05dbcf4637 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:47.451011 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b99128-6ab5-4d91-b9cb-2a05dbcf4637 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:47.452411 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 497/1008] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:47.458558 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d463ac-d67a-4e62-bf65-97d7dc789f09 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:47.458851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d463ac-d67a-4e62-bf65-97d7dc789f09 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:47.459133 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d463ac-d67a-4e62-bf65-97d7dc789f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:47.479617 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-c6d463ac-d67a-4e62-bf65-97d7dc789f09 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:47.481184 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-c6d463ac-d67a-4e62-bf65-97d7dc789f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:47.492737 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:47.553341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 512/1009] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:47 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:20:47.819870 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b99128-6ab5-4d91-b9cb-2a05dbcf4637 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:47.923367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b99128-6ab5-4d91-b9cb-2a05dbcf4637 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:47.958315 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999b44f-fa71-40ab-bbcd-14d3d3b6d749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:47.958941 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999b44f-fa71-40ab-bbcd-14d3d3b6d749 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:47.959215 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999b44f-fa71-40ab-bbcd-14d3d3b6d749 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:47.959480 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999b44f-fa71-40ab-bbcd-14d3d3b6d749 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:47.960053 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d999b44f-fa71-40ab-bbcd-14d3d3b6d749 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:47.961553 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d999b44f-fa71-40ab-bbcd-14d3d3b6d749 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:47.970425 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-d999b44f-fa71-40ab-bbcd-14d3d3b6d749 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 10:20:47.970425 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:47.970425 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 10:20:47.970425 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 10:20:47.970425 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 10:20:47.970425 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 10:20:47.970425 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 10:20:47.970425 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:47.970425 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 10:20:47.970425 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:47.970425 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 10:20:47.970425 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:47.970425 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:47.970425 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:47.970425 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:47.970425 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:47.970425 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:47.971422 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:47.971422 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:47.971422 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:47.971422 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:47.971422 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:47.971422 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:47.971422 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 10:20:47.971422 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 10:20:47.971422 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 10:20:47.971422 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 10:20:47.971422 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:47.971422 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:47.971422 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:47.971422 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:47.971422 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:47.971422 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:47.972300 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:47.972300 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:47.972300 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:47.972300 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:47.972300 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:47.972300 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:47.972300 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:47.972300 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:47.972300 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:47.972300 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:47.972300 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 10:20:47.972300 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:47.972300 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 10:20:47.972300 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:47.972300 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 498/1010] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:20:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:47.977533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d463ac-d67a-4e62-bf65-97d7dc789f09 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:48.080108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d463ac-d67a-4e62-bf65-97d7dc789f09 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:48.114230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c58042-d9a5-4f8a-9cce-b8ad266908a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:48.114837 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c58042-d9a5-4f8a-9cce-b8ad266908a7 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:48.115122 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c58042-d9a5-4f8a-9cce-b8ad266908a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:48.115398 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c58042-d9a5-4f8a-9cce-b8ad266908a7 None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:48.116023 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c58042-d9a5-4f8a-9cce-b8ad266908a7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:48.116397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c58042-d9a5-4f8a-9cce-b8ad266908a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:48.117755 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c58042-d9a5-4f8a-9cce-b8ad266908a7 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:48.127704 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 513/1011] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Sep 26 10:20:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:48.129008 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999b44f-fa71-40ab-bbcd-14d3d3b6d749 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:48.189600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:48.236300 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999b44f-fa71-40ab-bbcd-14d3d3b6d749 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:48.269732 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3620ed4-649e-4715-bf69-033ba9629df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:48.270355 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3620ed4-649e-4715-bf69-033ba9629df1 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:48.270635 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3620ed4-649e-4715-bf69-033ba9629df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:48.270906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3620ed4-649e-4715-bf69-033ba9629df1 None admin] PATH_INFO: `/v3/services` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:48.271523 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3620ed4-649e-4715-bf69-033ba9629df1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:48.271824 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3620ed4-649e-4715-bf69-033ba9629df1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:48.273185 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3620ed4-649e-4715-bf69-033ba9629df1 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:48.278052 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 499/1012] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Sep 26 10:20:48 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:48.283608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c58042-d9a5-4f8a-9cce-b8ad266908a7 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:48.296711 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:48.383906 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c58042-d9a5-4f8a-9cce-b8ad266908a7 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:48.417355 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87625af8-3264-4b35-823f-f15a171ee92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:48.417959 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87625af8-3264-4b35-823f-f15a171ee92b None admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:48.418263 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87625af8-3264-4b35-823f-f15a171ee92b None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:48.418536 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87625af8-3264-4b35-823f-f15a171ee92b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:48.419757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87625af8-3264-4b35-823f-f15a171ee92b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:48.421137 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87625af8-3264-4b35-823f-f15a171ee92b None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:48.442130 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 514/1013] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:20:48 2022] POST /identity/v3/endpoints => generated 334 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:20:49.444676 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28175d32-8ac6-48f6-8d39-765c0682ddfc None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:49.445198 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28175d32-8ac6-48f6-8d39-765c0682ddfc None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:49.445198 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28175d32-8ac6-48f6-8d39-765c0682ddfc None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:49.446707 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 500/1014] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:49.452320 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196eee41-cd3f-464e-a248-58b0d6028dec None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:49.452703 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196eee41-cd3f-464e-a248-58b0d6028dec None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:49.452861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196eee41-cd3f-464e-a248-58b0d6028dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:49.471236 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-196eee41-cd3f-464e-a248-58b0d6028dec None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:49.472409 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-196eee41-cd3f-464e-a248-58b0d6028dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:49.487379 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe53400-e580-4b1a-b6a6-9fba797f3d77 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:49.487660 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe53400-e580-4b1a-b6a6-9fba797f3d77 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:49.487931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe53400-e580-4b1a-b6a6-9fba797f3d77 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:49.489371 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 501/1015] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:20:49.494226 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8895eb-07be-4978-ac12-8e4d07fa8680 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:49.494508 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8895eb-07be-4978-ac12-8e4d07fa8680 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:49.494779 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8895eb-07be-4978-ac12-8e4d07fa8680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:49.513605 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-3c8895eb-07be-4978-ac12-8e4d07fa8680 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:49.514975 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-3c8895eb-07be-4978-ac12-8e4d07fa8680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:49.532878 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:49.549941 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 515/1016] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:49 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:20:49.588584 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 502/1017] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:49 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 10:20:49.820981 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196eee41-cd3f-464e-a248-58b0d6028dec None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:49.853274 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8895eb-07be-4978-ac12-8e4d07fa8680 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:49.905738 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:49.924192 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196eee41-cd3f-464e-a248-58b0d6028dec None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:49.958918 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8895eb-07be-4978-ac12-8e4d07fa8680 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:49.959073 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd7778-6851-48ed-9e12-aa03dba5ad69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:49.959700 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd7778-6851-48ed-9e12-aa03dba5ad69 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:49.959973 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd7778-6851-48ed-9e12-aa03dba5ad69 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:49.960304 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd7778-6851-48ed-9e12-aa03dba5ad69 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:49.961910 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffd7778-6851-48ed-9e12-aa03dba5ad69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:49.962769 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffd7778-6851-48ed-9e12-aa03dba5ad69 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:49.964535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 516/1018] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:49.970375 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd7778-6851-48ed-9e12-aa03dba5ad69 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:49.996661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95f817-b8fd-4055-9573-3c4fcb18acd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:49.997294 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95f817-b8fd-4055-9573-3c4fcb18acd2 None admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:49.997571 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95f817-b8fd-4055-9573-3c4fcb18acd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:49.997841 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95f817-b8fd-4055-9573-3c4fcb18acd2 None admin] PATH_INFO: `/v3/services` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:49.998514 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c95f817-b8fd-4055-9573-3c4fcb18acd2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:49.999858 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c95f817-b8fd-4055-9573-3c4fcb18acd2 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:50.022091 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 503/1019] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Sep 26 10:20:49 2022] POST /identity/v3/services => generated 242 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:20:50.043960 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:50.092979 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd7778-6851-48ed-9e12-aa03dba5ad69 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:50.136670 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74806f18-35a3-415a-a848-31222854cc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:50.137285 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74806f18-35a3-415a-a848-31222854cc61 None admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:50.137562 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74806f18-35a3-415a-a848-31222854cc61 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:50.137937 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74806f18-35a3-415a-a848-31222854cc61 None admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:50.138791 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74806f18-35a3-415a-a848-31222854cc61 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:50.140264 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74806f18-35a3-415a-a848-31222854cc61 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:50.173558 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 517/1020] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Sep 26 10:20:49 2022] POST /identity/v3/projects => generated 311 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:20:50.693213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7fa57-8be1-4be9-b9e4-ac5ea9eb337c None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:50.693730 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7fa57-8be1-4be9-b9e4-ac5ea9eb337c None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:50.693730 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7fa57-8be1-4be9-b9e4-ac5ea9eb337c None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:50.695163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 504/1021] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:50.700825 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c76ef2-39fe-48f7-a03a-d4fb5c4efb9f None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:50.701234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c76ef2-39fe-48f7-a03a-d4fb5c4efb9f None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:50.701411 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c76ef2-39fe-48f7-a03a-d4fb5c4efb9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:50.720306 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-e2c76ef2-39fe-48f7-a03a-d4fb5c4efb9f None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:50.721506 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-e2c76ef2-39fe-48f7-a03a-d4fb5c4efb9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:50.788366 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 518/1022] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:50 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:20:51.054649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7fa57-8be1-4be9-b9e4-ac5ea9eb337c None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:51.158934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7fa57-8be1-4be9-b9e4-ac5ea9eb337c None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:51.169310 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:51.194309 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee301b2-f38e-4921-8373-e808e30e903a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:51.195165 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee301b2-f38e-4921-8373-e808e30e903a None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:51.195593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee301b2-f38e-4921-8373-e808e30e903a None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:51.196019 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee301b2-f38e-4921-8373-e808e30e903a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:51.207446 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee301b2-f38e-4921-8373-e808e30e903a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:20:51.212396 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-eee301b2-f38e-4921-8373-e808e30e903a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 10:20:51.212396 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:51.212396 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:51.212396 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:51.212396 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:51.212396 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:51.212396 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:51.212396 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:51.212396 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:51.212396 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:51.212396 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:51.212396 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:51.212396 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:20:51.212396 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:20:51.212396 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:20:51.212396 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:20:51.213535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:20:51.213535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:20:51.213535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:20:51.213535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:20:51.213535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:51.213535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:51.213535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:51.213535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:51.213535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:51.213535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:51.213535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:51.213535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:51.213535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:51.213535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:51.213535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:51.213535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:51.214537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:51.214537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:51.214537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:51.214537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:51.214537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:20:51.214537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:20:51.214537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:20:51.214537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:20:51.214537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:20:51.214537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:20:51.214537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 10:20:51.214537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:51.218169 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 505/1023] 10.222.0.49 () {56 vars in 1120 bytes} [Mon Sep 26 10:20:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:51.221796 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c76ef2-39fe-48f7-a03a-d4fb5c4efb9f None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:51.264959 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:51.326519 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c76ef2-39fe-48f7-a03a-d4fb5c4efb9f None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:51.361208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6322b-73f0-429e-b0be-297465ece267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:51.361812 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6322b-73f0-429e-b0be-297465ece267 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:51.362118 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6322b-73f0-429e-b0be-297465ece267 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:51.362391 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6322b-73f0-429e-b0be-297465ece267 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:51.363045 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e6322b-73f0-429e-b0be-297465ece267 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:51.363347 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e6322b-73f0-429e-b0be-297465ece267 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:51.364738 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e6322b-73f0-429e-b0be-297465ece267 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:51.377897 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 519/1024] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Sep 26 10:20:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:51.379159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee301b2-f38e-4921-8373-e808e30e903a None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:51.498629 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee301b2-f38e-4921-8373-e808e30e903a None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:51.532450 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc54d6c7-60f0-48bd-8b0b-04a047535d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:51.533080 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54d6c7-60f0-48bd-8b0b-04a047535d72 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:51.533351 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54d6c7-60f0-48bd-8b0b-04a047535d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:51.533621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54d6c7-60f0-48bd-8b0b-04a047535d72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:51.544032 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc54d6c7-60f0-48bd-8b0b-04a047535d72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:51.545463 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc54d6c7-60f0-48bd-8b0b-04a047535d72 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:51.559289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-bc54d6c7-60f0-48bd-8b0b-04a047535d72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:51.559289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:51.559289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:51.559289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:51.559289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:51.559289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:51.559289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:51.559289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:51.559289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:51.559289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:51.559289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:51.559289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:51.559289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:51.559289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:51.559289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:51.559289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:51.560764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:51.560764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:51.560764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:51.560764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:51.560764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:51.560764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:51.560764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:51.560764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:51.560764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:51.560764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:51.560764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:51.560764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:51.560764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:51.560764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:51.560764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:51.560764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:51.560764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:51.561723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:51.561723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:51.561723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:51.562670 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 506/1025] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:51.566736 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6322b-73f0-429e-b0be-297465ece267 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:51.667326 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:51.671335 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6322b-73f0-429e-b0be-297465ece267 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:51.706623 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba14ac-e55a-49e0-8c2d-cc01727c6eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:51.707493 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba14ac-e55a-49e0-8c2d-cc01727c6eeb None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:51.707751 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba14ac-e55a-49e0-8c2d-cc01727c6eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:51.708030 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba14ac-e55a-49e0-8c2d-cc01727c6eeb None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:51.708668 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ba14ac-e55a-49e0-8c2d-cc01727c6eeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:51.708973 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ba14ac-e55a-49e0-8c2d-cc01727c6eeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:51.710372 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ba14ac-e55a-49e0-8c2d-cc01727c6eeb None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:51.726826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 520/1026] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:51.728013 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc54d6c7-60f0-48bd-8b0b-04a047535d72 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:51.753993 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:51.840459 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc54d6c7-60f0-48bd-8b0b-04a047535d72 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:51.875115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24f646-4f51-41d5-b59c-09e28a982325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:51.875747 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24f646-4f51-41d5-b59c-09e28a982325 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:51.876029 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24f646-4f51-41d5-b59c-09e28a982325 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:51.876301 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24f646-4f51-41d5-b59c-09e28a982325 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:51.884003 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d24f646-4f51-41d5-b59c-09e28a982325 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:51.885440 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d24f646-4f51-41d5-b59c-09e28a982325 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:51.896682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-1d24f646-4f51-41d5-b59c-09e28a982325 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 10:20:51.896682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:51.896682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:51.896682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:51.896682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:51.896682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:51.896682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:51.896682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:51.896682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:51.896682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:51.896682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:51.896682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:51.896682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:20:51.896682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:20:51.896682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:20:51.896682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:20:51.897723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:51.897723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:51.897723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:20:51.897723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:51.897723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:20:51.897723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:51.897723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:51.897723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:51.897723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:51.897723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:51.897723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:51.897723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:51.897723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:51.897723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:51.897723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:51.897723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:51.897723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:51.897723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:51.903837 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:51.903837 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:51.903837 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:20:51.903837 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:20:51.903837 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:20:51.903837 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:20:51.903837 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:20:51.903837 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:20:51.903837 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 10:20:51.903837 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:51.903837 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 507/1027] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Sep 26 10:20:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:20:51.904451 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba14ac-e55a-49e0-8c2d-cc01727c6eeb None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:52.006246 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba14ac-e55a-49e0-8c2d-cc01727c6eeb None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:52.040197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b1f25-ff48-4b4e-b13a-f85856e37efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:52.040867 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b1f25-ff48-4b4e-b13a-f85856e37efb None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:52.041149 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b1f25-ff48-4b4e-b13a-f85856e37efb None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:52.041427 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b1f25-ff48-4b4e-b13a-f85856e37efb None admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:52.042236 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b1f25-ff48-4b4e-b13a-f85856e37efb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:52.042544 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b1f25-ff48-4b4e-b13a-f85856e37efb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:52.044488 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b1f25-ff48-4b4e-b13a-f85856e37efb None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:52.066975 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 521/1028] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Sep 26 10:20:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:52.068162 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24f646-4f51-41d5-b59c-09e28a982325 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:52.157459 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:52.178287 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24f646-4f51-41d5-b59c-09e28a982325 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:52.212436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c291d2-952e-46bd-935d-d3303926b193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:52.213045 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c291d2-952e-46bd-935d-d3303926b193 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:52.213326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c291d2-952e-46bd-935d-d3303926b193 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:52.213598 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c291d2-952e-46bd-935d-d3303926b193 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:52.252502 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33335ab1-40a3-4af7-b072-ade10922df45 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:52.252850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33335ab1-40a3-4af7-b072-ade10922df45 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:52.253065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33335ab1-40a3-4af7-b072-ade10922df45 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:52.254708 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 522/1029] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:20:52.262166 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb0f23-6072-4d73-86c4-21456c9d170c None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:52.262166 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb0f23-6072-4d73-86c4-21456c9d170c None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:52.262432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb0f23-6072-4d73-86c4-21456c9d170c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:52.283644 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-8ecb0f23-6072-4d73-86c4-21456c9d170c None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:52.284844 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-8ecb0f23-6072-4d73-86c4-21456c9d170c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:52.299673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:52.300938 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c291d2-952e-46bd-935d-d3303926b193 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:52.302635 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c291d2-952e-46bd-935d-d3303926b193 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:52.358054 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 523/1030] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 10:20:52.368317 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b26eca-26ba-4758-82fd-d1d41911e828 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:52.368601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b26eca-26ba-4758-82fd-d1d41911e828 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:52.368878 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b26eca-26ba-4758-82fd-d1d41911e828 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:52.370191 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 524/1031] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:20:52.376516 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747df76-b59b-4568-9477-14e20101ac29 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:52.376783 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747df76-b59b-4568-9477-14e20101ac29 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:52.377052 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747df76-b59b-4568-9477-14e20101ac29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:52.393482 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:52.395004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-b747df76-b59b-4568-9477-14e20101ac29 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:52.395909 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-b747df76-b59b-4568-9477-14e20101ac29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:52.415614 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 508/1032] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:20:52 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 10:20:52.421645 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c291d2-952e-46bd-935d-d3303926b193 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:52.466951 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 525/1033] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:20:52.522917 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c291d2-952e-46bd-935d-d3303926b193 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:52.557275 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcae8c9-18fb-49bb-b23a-f4313b9badf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:52.557897 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcae8c9-18fb-49bb-b23a-f4313b9badf7 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:52.558205 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcae8c9-18fb-49bb-b23a-f4313b9badf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:52.558478 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcae8c9-18fb-49bb-b23a-f4313b9badf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:52.568763 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcae8c9-18fb-49bb-b23a-f4313b9badf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:52.570230 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcae8c9-18fb-49bb-b23a-f4313b9badf7 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:52.585631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-4fcae8c9-18fb-49bb-b23a-f4313b9badf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:52.585631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:52.585631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:52.585631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:52.585631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:52.585631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:52.585631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:52.585631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:52.585631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:52.585631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:52.585631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:52.585631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:52.585631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:52.585631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:52.585631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:52.585631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:52.586744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:52.586744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:52.586744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:52.586744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:52.586744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:52.586744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:52.586744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:52.586744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:52.586744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:52.586744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:52.586744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:52.586744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:52.586744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:52.586744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:52.586744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:52.586744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:52.586744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:52.591822 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:52.591822 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:52.591822 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:52.591822 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 509/1034] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:20:52.596240 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b747df76-b59b-4568-9477-14e20101ac29 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:52.638122 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcae8c9-18fb-49bb-b23a-f4313b9badf7 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:52.675240 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:52.675935 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:52.698539 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b747df76-b59b-4568-9477-14e20101ac29 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:52.736690 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c69dab8-d72a-4d6d-a96a-2fd88e541c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:52.737369 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69dab8-d72a-4d6d-a96a-2fd88e541c7b None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:52.737559 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69dab8-d72a-4d6d-a96a-2fd88e541c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:52.737820 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69dab8-d72a-4d6d-a96a-2fd88e541c7b None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:52.738492 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c69dab8-d72a-4d6d-a96a-2fd88e541c7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:52.738799 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c69dab8-d72a-4d6d-a96a-2fd88e541c7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:52.740203 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c69dab8-d72a-4d6d-a96a-2fd88e541c7b None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:52.748321 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcae8c9-18fb-49bb-b23a-f4313b9badf7 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:52.752253 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 526/1035] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:52.753694 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c69dab8-d72a-4d6d-a96a-2fd88e541c7b None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:52.783675 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f650c18-fc6e-43ee-949b-155cfd9d523c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:52.784225 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f650c18-fc6e-43ee-949b-155cfd9d523c None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:52.784510 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f650c18-fc6e-43ee-949b-155cfd9d523c None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:52.784789 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f650c18-fc6e-43ee-949b-155cfd9d523c None admin] PATH_INFO: `/v3/services/image` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:52.785363 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f650c18-fc6e-43ee-949b-155cfd9d523c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:52.786752 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f650c18-fc6e-43ee-949b-155cfd9d523c None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:52.795475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-7f650c18-fc6e-43ee-949b-155cfd9d523c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 10:20:52.795475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:52.795475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 10:20:52.795475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 10:20:52.795475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 10:20:52.795475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 10:20:52.795475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 10:20:52.795475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:52.795475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 10:20:52.795475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:52.795475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 10:20:52.795475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:52.795475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:52.795475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:52.795475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:52.795475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:52.795475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:52.796689 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:52.796689 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:52.796689 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:52.796689 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:52.796689 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:52.796689 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:52.796689 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 10:20:52.796689 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 10:20:52.796689 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 10:20:52.796689 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 10:20:52.796689 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:52.796689 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:52.796689 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:52.796689 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:52.796689 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:52.796689 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:52.797750 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:52.797750 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:52.797750 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:52.797750 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:52.797750 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:52.797750 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:52.797750 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:52.797750 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:52.797750 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:52.797750 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:52.797750 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 10:20:52.797750 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:52.797750 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 10:20:52.797750 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:52.799243 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 510/1036] 10.222.0.49 () {56 vars in 1105 bytes} [Mon Sep 26 10:20:52 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:20:52.800454 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f650c18-fc6e-43ee-949b-155cfd9d523c None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:52.857283 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c69dab8-d72a-4d6d-a96a-2fd88e541c7b None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:52.890489 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd00daf-2e4f-45b5-8209-d7195c386e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:52.891109 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd00daf-2e4f-45b5-8209-d7195c386e1b None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:52.891382 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd00daf-2e4f-45b5-8209-d7195c386e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:52.891650 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd00daf-2e4f-45b5-8209-d7195c386e1b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:52.892736 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd00daf-2e4f-45b5-8209-d7195c386e1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:52.894175 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd00daf-2e4f-45b5-8209-d7195c386e1b None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:52.895722 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 527/1037] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:52.896462 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd00daf-2e4f-45b5-8209-d7195c386e1b None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:52.898234 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:52.902905 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f650c18-fc6e-43ee-949b-155cfd9d523c None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:52.905687 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:52.937915 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36787f00-f26c-49e9-a3e5-e0554f0df267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:52.938562 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36787f00-f26c-49e9-a3e5-e0554f0df267 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:52.938844 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36787f00-f26c-49e9-a3e5-e0554f0df267 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:52.939115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36787f00-f26c-49e9-a3e5-e0554f0df267 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:52.945644 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36787f00-f26c-49e9-a3e5-e0554f0df267 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:52.947112 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36787f00-f26c-49e9-a3e5-e0554f0df267 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:52.958986 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-36787f00-f26c-49e9-a3e5-e0554f0df267 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:52.958986 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:52.958986 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:52.958986 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:52.958986 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:52.958986 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:52.958986 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:52.958986 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:52.958986 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:52.958986 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:52.958986 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:52.958986 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:52.958986 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:20:52.958986 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:20:52.958986 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:20:52.959875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:20:52.959875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:52.959875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:52.959875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:52.959875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:52.959875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:52.959875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:52.959875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:52.959875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:52.959875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:52.959875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:52.959875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:52.959875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:52.959875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:52.959875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:52.959875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:52.959875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:52.960850 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:20:52.960850 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:20:52.960850 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:20:52.960850 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:20:52.960850 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:20:52.960850 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:20:52.960850 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:52.960850 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:52.960850 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 511/1038] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:20:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:52.960850 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36787f00-f26c-49e9-a3e5-e0554f0df267 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:52.997840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd00daf-2e4f-45b5-8209-d7195c386e1b None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:53.032268 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd3e67-3e17-48cc-a557-56f464f493b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:53.032865 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd3e67-3e17-48cc-a557-56f464f493b4 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:53.033141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd3e67-3e17-48cc-a557-56f464f493b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:53.033413 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd3e67-3e17-48cc-a557-56f464f493b4 None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:53.034098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd3e67-3e17-48cc-a557-56f464f493b4 None admin] RBAC: Adding query filter params (name=image) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:53.034355 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd3e67-3e17-48cc-a557-56f464f493b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:53.036100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd3e67-3e17-48cc-a557-56f464f493b4 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:53.041228 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 528/1039] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:53.041986 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd3e67-3e17-48cc-a557-56f464f493b4 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:53.065329 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36787f00-f26c-49e9-a3e5-e0554f0df267 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:53.099801 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998469d4-91f7-4253-8cbf-c97cd1f22fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:53.100398 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998469d4-91f7-4253-8cbf-c97cd1f22fe6 None admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:53.100679 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998469d4-91f7-4253-8cbf-c97cd1f22fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:53.100950 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998469d4-91f7-4253-8cbf-c97cd1f22fe6 None admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:53.101596 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998469d4-91f7-4253-8cbf-c97cd1f22fe6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:53.103047 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998469d4-91f7-4253-8cbf-c97cd1f22fe6 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:53.108472 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:53.124113 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 512/1040] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 10:20:52 2022] POST /identity/v3/users => generated 285 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:20:53.124924 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998469d4-91f7-4253-8cbf-c97cd1f22fe6 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:53.142472 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:53.148154 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd3e67-3e17-48cc-a557-56f464f493b4 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:53.187516 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d8d7f-9e8c-4990-9d33-b91286062c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:53.188146 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d8d7f-9e8c-4990-9d33-b91286062c63 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:53.188420 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d8d7f-9e8c-4990-9d33-b91286062c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:53.188694 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d8d7f-9e8c-4990-9d33-b91286062c63 None admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:53.189422 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447d8d7f-9e8c-4990-9d33-b91286062c63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:53.189706 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447d8d7f-9e8c-4990-9d33-b91286062c63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:53.191207 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447d8d7f-9e8c-4990-9d33-b91286062c63 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:53.210475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 529/1041] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:20:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:53.211994 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d8d7f-9e8c-4990-9d33-b91286062c63 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:53.236302 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998469d4-91f7-4253-8cbf-c97cd1f22fe6 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:53.277170 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c1daa-a99a-4078-bbfd-423d2728955b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:53.277796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c1daa-a99a-4078-bbfd-423d2728955b None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:53.278092 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c1daa-a99a-4078-bbfd-423d2728955b None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:53.278399 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c1daa-a99a-4078-bbfd-423d2728955b None admin] PATH_INFO: `/v3/services` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:53.279040 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c1daa-a99a-4078-bbfd-423d2728955b None admin] RBAC: Adding query filter params (type=image) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:53.279356 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c1daa-a99a-4078-bbfd-423d2728955b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:53.280761 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c1daa-a99a-4078-bbfd-423d2728955b None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:53.286356 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 513/1042] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:53 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:53.292146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c1daa-a99a-4078-bbfd-423d2728955b None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:53.327712 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d8d7f-9e8c-4990-9d33-b91286062c63 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:53.363774 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:53.373815 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7069ee4b-5f4d-4f79-94a3-72f3e76a12cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:53.374736 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069ee4b-5f4d-4f79-94a3-72f3e76a12cb None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:53.375010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069ee4b-5f4d-4f79-94a3-72f3e76a12cb None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:53.375278 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069ee4b-5f4d-4f79-94a3-72f3e76a12cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:53.376005 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7069ee4b-5f4d-4f79-94a3-72f3e76a12cb None admin] RBAC: Adding query filter params (role.id=dabb7e453c8742aa809ef7543c15d1d3, scope.project.id=cff3a6011f4c4893bddffdad3d69f3df, user.id=cee41014d5d043888c6b399710bccab4) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:53.376308 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7069ee4b-5f4d-4f79-94a3-72f3e76a12cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:53.377726 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7069ee4b-5f4d-4f79-94a3-72f3e76a12cb None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:53.383993 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 530/1043] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:20:53 2022] GET /identity/v3/role_assignments?user.id=cee41014d5d043888c6b399710bccab4&scope.project.id=cff3a6011f4c4893bddffdad3d69f3df&role.id=dabb7e453c8742aa809ef7543c15d1d3 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:53.388471 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:53.415208 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c1daa-a99a-4078-bbfd-423d2728955b None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:53.456656 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cba003-eabe-4da4-a9c9-2fc442447789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:53.456656 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cba003-eabe-4da4-a9c9-2fc442447789 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:53.457098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cba003-eabe-4da4-a9c9-2fc442447789 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:53.457789 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cba003-eabe-4da4-a9c9-2fc442447789 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:53.459088 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cba003-eabe-4da4-a9c9-2fc442447789 None admin] RBAC: Adding query filter params (interface=public, service_id=b17ae696d8754feb8d61499c2162b48a, region_id=RegionOne) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:53.461222 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cba003-eabe-4da4-a9c9-2fc442447789 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:53.461222 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cba003-eabe-4da4-a9c9-2fc442447789 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:53.468610 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 514/1044] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Sep 26 10:20:53 2022] GET /identity/v3/endpoints?service_id=b17ae696d8754feb8d61499c2162b48a&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:53.473280 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7069ee4b-5f4d-4f79-94a3-72f3e76a12cb None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:53.581560 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7069ee4b-5f4d-4f79-94a3-72f3e76a12cb None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:53.615359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c6e62d-9731-4f9d-a6cc-d3ddae96706d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:53.615962 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6e62d-9731-4f9d-a6cc-d3ddae96706d None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:53.616237 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6e62d-9731-4f9d-a6cc-d3ddae96706d None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:53.616505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6e62d-9731-4f9d-a6cc-d3ddae96706d None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:53.617098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c6e62d-9731-4f9d-a6cc-d3ddae96706d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:53.618494 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c6e62d-9731-4f9d-a6cc-d3ddae96706d None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:53.625931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 531/1045] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Sep 26 10:20:53 2022] GET /identity/v3/services => generated 1957 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:20:55.360566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40349828-c92f-4c88-b951-d44756a109bd None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:55.361101 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40349828-c92f-4c88-b951-d44756a109bd None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:55.361101 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40349828-c92f-4c88-b951-d44756a109bd None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:55.363034 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 515/1046] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:20:55.368150 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44436a72-7644-43a1-99a0-c85b17c1b948 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:55.368590 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44436a72-7644-43a1-99a0-c85b17c1b948 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:55.368782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44436a72-7644-43a1-99a0-c85b17c1b948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:55.387282 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-44436a72-7644-43a1-99a0-c85b17c1b948 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:55.389066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-44436a72-7644-43a1-99a0-c85b17c1b948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:55.447626 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:55.464473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 532/1047] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 10:20:55.563787 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879572a-421d-4932-9614-84e4958948e6 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:55.564064 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879572a-421d-4932-9614-84e4958948e6 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:55.564525 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879572a-421d-4932-9614-84e4958948e6 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:55.565999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 516/1048] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:55.571093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbb02f-906c-4114-a1e9-e99079c0fa44 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:55.571542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbb02f-906c-4114-a1e9-e99079c0fa44 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:55.571970 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbb02f-906c-4114-a1e9-e99079c0fa44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:55.590619 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-57fbb02f-906c-4114-a1e9-e99079c0fa44 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:55.592039 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-57fbb02f-906c-4114-a1e9-e99079c0fa44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:55.661607 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 533/1049] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:20:55.737677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879572a-421d-4932-9614-84e4958948e6 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:55.767177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e203454-5e7e-4098-8a7a-6eb7a2f9b6c5 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:55.767458 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e203454-5e7e-4098-8a7a-6eb7a2f9b6c5 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:55.767933 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e203454-5e7e-4098-8a7a-6eb7a2f9b6c5 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:55.769726 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 534/1050] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:20:55.774494 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b88745-1748-4bd0-8a81-4c5c7b1d303a None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:55.774772 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b88745-1748-4bd0-8a81-4c5c7b1d303a None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:55.775045 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b88745-1748-4bd0-8a81-4c5c7b1d303a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:55.794584 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-b4b88745-1748-4bd0-8a81-4c5c7b1d303a None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:55.795779 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-b4b88745-1748-4bd0-8a81-4c5c7b1d303a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:55.854428 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879572a-421d-4932-9614-84e4958948e6 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:55.871116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 535/1051] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:20:55.888894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca3888-dca0-4aca-be23-36b817244941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:55.889516 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca3888-dca0-4aca-be23-36b817244941 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:55.889787 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca3888-dca0-4aca-be23-36b817244941 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:55.890076 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca3888-dca0-4aca-be23-36b817244941 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:55.900646 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ca3888-dca0-4aca-be23-36b817244941 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:20:55.905436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-66ca3888-dca0-4aca-be23-36b817244941 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:20:55.905436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:55.905436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:55.905436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:55.905436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:55.905436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:55.905436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:55.905436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:55.905436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:55.905436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:55.905436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:55.905436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:55.905436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:20:55.905436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:20:55.905436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:20:55.905436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:20:55.906671 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:20:55.906671 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:20:55.906671 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:20:55.906671 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:20:55.906671 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:55.906671 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:55.906671 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:55.906671 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:55.906671 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:55.906671 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:55.906671 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:55.906671 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:55.906671 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:55.906671 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:55.906671 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:55.906671 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:55.907511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:55.907511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:55.907511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:55.907511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:55.907511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:20:55.907511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:20:55.907511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:20:55.907511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:20:55.907511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:20:55.907511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:20:55.907511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:20:55.907511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:55.907511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 517/1052] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:20:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:55.912551 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b88745-1748-4bd0-8a81-4c5c7b1d303a None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:55.936416 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca3888-dca0-4aca-be23-36b817244941 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:55.939294 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:55.968575 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:56.026499 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b88745-1748-4bd0-8a81-4c5c7b1d303a None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:56.043098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca3888-dca0-4aca-be23-36b817244941 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:56.070016 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981dabaa-aa42-4fdd-a107-45f5d70924d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:56.070656 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dabaa-aa42-4fdd-a107-45f5d70924d0 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:56.071096 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dabaa-aa42-4fdd-a107-45f5d70924d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:56.071253 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dabaa-aa42-4fdd-a107-45f5d70924d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:56.072091 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981dabaa-aa42-4fdd-a107-45f5d70924d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:56.072091 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981dabaa-aa42-4fdd-a107-45f5d70924d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:56.073468 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981dabaa-aa42-4fdd-a107-45f5d70924d0 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:56.081377 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb55613-2c64-42a8-9d39-d7274e5e8fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:56.081997 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb55613-2c64-42a8-9d39-d7274e5e8fd1 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:56.082543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb55613-2c64-42a8-9d39-d7274e5e8fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:56.083008 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb55613-2c64-42a8-9d39-d7274e5e8fd1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:56.083131 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 536/1053] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:20:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:56.088655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981dabaa-aa42-4fdd-a107-45f5d70924d0 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:56.097806 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb55613-2c64-42a8-9d39-d7274e5e8fd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:20:56.103048 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-bcb55613-2c64-42a8-9d39-d7274e5e8fd1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 10:20:56.103048 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:56.103048 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:56.103048 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:56.103048 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:56.103048 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:56.103048 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:56.103048 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:56.103048 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:56.103048 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:56.103048 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:56.103048 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:56.103048 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:20:56.103048 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:20:56.103048 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:20:56.103048 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:20:56.104066 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:20:56.104066 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:20:56.104066 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:20:56.104066 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:20:56.104066 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:56.104066 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:56.104066 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:56.104066 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:56.104066 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:56.104066 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:56.104066 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:56.104066 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:56.104066 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:56.104066 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:56.104066 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:56.104066 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:56.107280 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:56.107280 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:56.107280 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:56.107280 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:56.107280 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:20:56.107280 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:20:56.107280 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:20:56.107280 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:20:56.107280 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:20:56.107280 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:20:56.107280 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 10:20:56.107280 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:20:56.107280 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 518/1054] 10.222.0.49 () {56 vars in 1120 bytes} [Mon Sep 26 10:20:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:20:56.109857 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb55613-2c64-42a8-9d39-d7274e5e8fd1 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:56.173784 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:56.196032 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981dabaa-aa42-4fdd-a107-45f5d70924d0 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:56.206874 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:56.214237 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb55613-2c64-42a8-9d39-d7274e5e8fd1 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:56.230943 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978eb49-f5ff-45a5-9588-3abe1784732e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:56.231615 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978eb49-f5ff-45a5-9588-3abe1784732e None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:56.231892 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978eb49-f5ff-45a5-9588-3abe1784732e None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:56.232164 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978eb49-f5ff-45a5-9588-3abe1784732e None admin] PATH_INFO: `/v3/users/25376c83a3934017873ce7cde98bf027` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:56.245247 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978eb49-f5ff-45a5-9588-3abe1784732e None admin] RBAC: Authorizing `identity:get_user(user_id=25376c83a3934017873ce7cde98bf027)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:56.246701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978eb49-f5ff-45a5-9588-3abe1784732e None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:56.247525 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eadd9b6-4f72-436a-81a6-2e9c1c3b16aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:56.248129 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eadd9b6-4f72-436a-81a6-2e9c1c3b16aa None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:56.248405 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eadd9b6-4f72-436a-81a6-2e9c1c3b16aa None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:56.248671 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eadd9b6-4f72-436a-81a6-2e9c1c3b16aa None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:56.253734 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 537/1055] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:20:56 2022] GET /identity/v3/users/25376c83a3934017873ce7cde98bf027 => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:56.253886 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eadd9b6-4f72-436a-81a6-2e9c1c3b16aa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:56.253886 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eadd9b6-4f72-436a-81a6-2e9c1c3b16aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:56.254058 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978eb49-f5ff-45a5-9588-3abe1784732e None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:56.254138 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eadd9b6-4f72-436a-81a6-2e9c1c3b16aa None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:56.259758 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 519/1056] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Sep 26 10:20:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:56.259758 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eadd9b6-4f72-436a-81a6-2e9c1c3b16aa None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:56.348578 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978eb49-f5ff-45a5-9588-3abe1784732e None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:56.355469 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eadd9b6-4f72-436a-81a6-2e9c1c3b16aa None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:56.377818 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:56.383148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d9992-7ff2-4bd7-b57e-0bbd55b68fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:56.383917 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d9992-7ff2-4bd7-b57e-0bbd55b68fe5 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:56.384193 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d9992-7ff2-4bd7-b57e-0bbd55b68fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:56.384767 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d9992-7ff2-4bd7-b57e-0bbd55b68fe5 None admin] PATH_INFO: `/v3/services/image` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:56.385344 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5d9992-7ff2-4bd7-b57e-0bbd55b68fe5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:56.386719 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5d9992-7ff2-4bd7-b57e-0bbd55b68fe5 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:56.389920 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e684426-a2d5-4eae-8ee5-fb9f50eea2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:56.390545 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e684426-a2d5-4eae-8ee5-fb9f50eea2af None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:56.390826 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e684426-a2d5-4eae-8ee5-fb9f50eea2af None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:56.391093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e684426-a2d5-4eae-8ee5-fb9f50eea2af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:56.395847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-6d5d9992-7ff2-4bd7-b57e-0bbd55b68fe5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 10:20:56.395847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:56.395847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 10:20:56.395847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 10:20:56.395847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 10:20:56.395847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 10:20:56.395847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 10:20:56.395847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:56.395847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 10:20:56.395847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:56.395847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 10:20:56.395847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:56.395847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:56.395847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:56.395847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:56.395847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:56.395847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:56.396823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:56.396823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:56.396823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:56.396823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:56.396823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:56.396823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:56.396823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 10:20:56.396823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 10:20:56.396823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 10:20:56.396823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 10:20:56.396823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:56.396823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:56.396823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:56.396823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:56.396823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:56.396823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:56.397691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:56.397691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:56.397691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:56.397691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:56.397691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:56.397691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:56.397691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:56.397691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:56.397691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:56.397691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:56.397691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 10:20:56.397691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:20:56.397691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 10:20:56.397691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:56.397691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 538/1057] 10.222.0.49 () {56 vars in 1105 bytes} [Mon Sep 26 10:20:56 2022] GET /identity/v3/services/image => generated 86 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:56.399868 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d9992-7ff2-4bd7-b57e-0bbd55b68fe5 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:56.430809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:56.458395 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e684426-a2d5-4eae-8ee5-fb9f50eea2af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:56.459845 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e684426-a2d5-4eae-8ee5-fb9f50eea2af None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:56.501361 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d9992-7ff2-4bd7-b57e-0bbd55b68fe5 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:56.534773 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e81b88-6cf3-4110-aae7-3dc761f9a787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:56.535404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e81b88-6cf3-4110-aae7-3dc761f9a787 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:56.535693 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e81b88-6cf3-4110-aae7-3dc761f9a787 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:56.535967 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e81b88-6cf3-4110-aae7-3dc761f9a787 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:56.546525 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e81b88-6cf3-4110-aae7-3dc761f9a787 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:56.547951 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e81b88-6cf3-4110-aae7-3dc761f9a787 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:56.557306 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 520/1058] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:20:56 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 10:20:56.558150 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e684426-a2d5-4eae-8ee5-fb9f50eea2af None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:56.561786 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-55e81b88-6cf3-4110-aae7-3dc761f9a787 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:56.561786 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:56.561786 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:56.561786 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:56.561786 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:56.561786 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:56.561786 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:56.561786 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:56.561786 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:56.561786 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:56.561786 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:56.561786 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:56.561786 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:56.561786 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:56.561786 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:56.561786 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:56.562762 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:56.562762 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:56.562762 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:56.562762 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:56.562762 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:56.562762 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:56.562762 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:56.562762 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:56.562762 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:56.562762 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:56.562762 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:56.562762 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:56.562762 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:56.562762 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:56.562762 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:56.562762 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:56.562762 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:56.563659 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:56.563659 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:56.563659 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:56.563659 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 539/1059] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:56.563888 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e81b88-6cf3-4110-aae7-3dc761f9a787 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:56.584227 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:56.651169 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:56.659093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e684426-a2d5-4eae-8ee5-fb9f50eea2af None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:56.667521 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e81b88-6cf3-4110-aae7-3dc761f9a787 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:56.692302 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384a473-0d7a-4b53-a7c5-9d709ffa3ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:56.692838 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384a473-0d7a-4b53-a7c5-9d709ffa3ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:56.693108 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384a473-0d7a-4b53-a7c5-9d709ffa3ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:56.693380 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384a473-0d7a-4b53-a7c5-9d709ffa3ec7 None admin] PATH_INFO: `/v3/services` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:56.693990 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a384a473-0d7a-4b53-a7c5-9d709ffa3ec7 None admin] RBAC: Adding query filter params (name=image) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:56.694317 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a384a473-0d7a-4b53-a7c5-9d709ffa3ec7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:56.695665 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a384a473-0d7a-4b53-a7c5-9d709ffa3ec7 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:56.700321 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 521/1060] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:56.701320 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa1cd1-efc5-4b88-b345-e62e6d39c235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:56.701572 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384a473-0d7a-4b53-a7c5-9d709ffa3ec7 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:56.701943 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa1cd1-efc5-4b88-b345-e62e6d39c235 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:56.702290 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa1cd1-efc5-4b88-b345-e62e6d39c235 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:56.702524 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa1cd1-efc5-4b88-b345-e62e6d39c235 None admin] PATH_INFO: `/v3/projects/4df4efbd6e0440b490af925e22b13188` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:56.703640 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa1cd1-efc5-4b88-b345-e62e6d39c235 None admin] RBAC: Authorizing `identity:get_project(project_id=4df4efbd6e0440b490af925e22b13188)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:56.705255 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa1cd1-efc5-4b88-b345-e62e6d39c235 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:56.712067 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 540/1061] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:20:56 2022] GET /identity/v3/projects/4df4efbd6e0440b490af925e22b13188 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:56.712067 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa1cd1-efc5-4b88-b345-e62e6d39c235 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:56.788091 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:56.801862 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384a473-0d7a-4b53-a7c5-9d709ffa3ec7 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:56.810624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa1cd1-efc5-4b88-b345-e62e6d39c235 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:56.837707 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8a9c4-679e-4d03-a84b-46e649652ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:56.838345 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8a9c4-679e-4d03-a84b-46e649652ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:56.838626 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8a9c4-679e-4d03-a84b-46e649652ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:56.838898 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8a9c4-679e-4d03-a84b-46e649652ea9 None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:56.839537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e8a9c4-679e-4d03-a84b-46e649652ea9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:56.839841 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e8a9c4-679e-4d03-a84b-46e649652ea9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:56.841239 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e8a9c4-679e-4d03-a84b-46e649652ea9 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:56.843954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2d3aa-3528-460b-9cb4-c3b5386a9f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:56.844557 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2d3aa-3528-460b-9cb4-c3b5386a9f78 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:56.844836 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2d3aa-3528-460b-9cb4-c3b5386a9f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:56.845108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2d3aa-3528-460b-9cb4-c3b5386a9f78 None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:56.845728 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db2d3aa-3528-460b-9cb4-c3b5386a9f78 None admin] RBAC: Adding query filter params (type=image) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:56.846047 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db2d3aa-3528-460b-9cb4-c3b5386a9f78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:56.847384 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db2d3aa-3528-460b-9cb4-c3b5386a9f78 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:56.852119 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 541/1062] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:56.852887 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2d3aa-3528-460b-9cb4-c3b5386a9f78 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:56.858461 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 522/1063] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:56.858461 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8a9c4-679e-4d03-a84b-46e649652ea9 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:56.871677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:56.952644 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2d3aa-3528-460b-9cb4-c3b5386a9f78 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:56.954896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8a9c4-679e-4d03-a84b-46e649652ea9 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:56.986724 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4332b-1c39-470e-8200-d3e1dec86253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:56.987322 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4332b-1c39-470e-8200-d3e1dec86253 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:56.987602 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4332b-1c39-470e-8200-d3e1dec86253 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:56.987875 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4332b-1c39-470e-8200-d3e1dec86253 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:56.988195 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a22094-9d24-425b-aa5d-bdc6eff82d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:56.988590 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac4332b-1c39-470e-8200-d3e1dec86253 None admin] RBAC: Adding query filter params (role.id=532b37d9076c45618cf352aafb85f35e, scope.project.id=4df4efbd6e0440b490af925e22b13188, user.id=25376c83a3934017873ce7cde98bf027) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:56.988788 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a22094-9d24-425b-aa5d-bdc6eff82d0e None admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:56.988887 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac4332b-1c39-470e-8200-d3e1dec86253 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:56.989068 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a22094-9d24-425b-aa5d-bdc6eff82d0e None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:56.989339 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a22094-9d24-425b-aa5d-bdc6eff82d0e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:56.989996 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a22094-9d24-425b-aa5d-bdc6eff82d0e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:56.990325 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac4332b-1c39-470e-8200-d3e1dec86253 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:56.991364 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a22094-9d24-425b-aa5d-bdc6eff82d0e None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:56.995672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 542/1064] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:20:56 2022] GET /identity/v3/role_assignments?user.id=25376c83a3934017873ce7cde98bf027&scope.project.id=4df4efbd6e0440b490af925e22b13188&role.id=532b37d9076c45618cf352aafb85f35e => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:56.996443 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4332b-1c39-470e-8200-d3e1dec86253 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:57.026056 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 523/1065] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:20:56 2022] POST /identity/v3/endpoints => generated 327 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:20:57.096043 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:57.119997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4332b-1c39-470e-8200-d3e1dec86253 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:57.167140 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee94fd46-d890-4fce-a3e2-a7a7fab75067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:57.167806 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94fd46-d890-4fce-a3e2-a7a7fab75067 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:57.168086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94fd46-d890-4fce-a3e2-a7a7fab75067 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:57.168356 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94fd46-d890-4fce-a3e2-a7a7fab75067 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:57.177323 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee94fd46-d890-4fce-a3e2-a7a7fab75067 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:57.178828 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee94fd46-d890-4fce-a3e2-a7a7fab75067 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:57.191396 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-ee94fd46-d890-4fce-a3e2-a7a7fab75067 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 10:20:57.191396 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:57.191396 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:57.191396 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:57.191396 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:57.191396 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:57.191396 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:57.191396 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:57.191396 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:57.191396 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:57.191396 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:57.191396 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:57.191396 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:20:57.191396 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:20:57.191396 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:20:57.191396 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:20:57.192412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:57.192412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:57.192412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:20:57.192412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:57.192412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:20:57.192412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:20:57.192412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:57.192412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:57.192412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:57.192412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:57.192412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:57.192412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:57.192412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:57.192412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:57.192412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:57.192412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:57.192412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:57.192412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:57.193366 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:57.193366 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:57.193366 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:20:57.193366 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:20:57.193366 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:20:57.193366 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:20:57.193366 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:20:57.193366 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:20:57.193366 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 10:20:57.193366 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:57.193366 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 543/1066] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Sep 26 10:20:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:57.198674 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a22094-9d24-425b-aa5d-bdc6eff82d0e None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:57.203263 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:57.307700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a22094-9d24-425b-aa5d-bdc6eff82d0e None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:57.343404 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74289ec1-365c-4752-ba1e-5138f0ff563d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:57.344020 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74289ec1-365c-4752-ba1e-5138f0ff563d None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:57.344296 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74289ec1-365c-4752-ba1e-5138f0ff563d None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:57.344575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74289ec1-365c-4752-ba1e-5138f0ff563d None admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:57.345306 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74289ec1-365c-4752-ba1e-5138f0ff563d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:57.345607 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74289ec1-365c-4752-ba1e-5138f0ff563d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:57.347062 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74289ec1-365c-4752-ba1e-5138f0ff563d None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:57.364625 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 524/1067] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Sep 26 10:20:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:57.370639 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee94fd46-d890-4fce-a3e2-a7a7fab75067 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:57.473037 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee94fd46-d890-4fce-a3e2-a7a7fab75067 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:57.508207 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21661953-c767-4264-888d-22b0c6fec32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:57.509193 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21661953-c767-4264-888d-22b0c6fec32b None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:57.509668 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21661953-c767-4264-888d-22b0c6fec32b None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:57.510216 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21661953-c767-4264-888d-22b0c6fec32b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:57.514263 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:57.523222 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21661953-c767-4264-888d-22b0c6fec32b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:57.524924 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21661953-c767-4264-888d-22b0c6fec32b None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:57.540528 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-21661953-c767-4264-888d-22b0c6fec32b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:57.540528 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:57.540528 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:57.540528 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:57.540528 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:57.540528 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:57.540528 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:57.540528 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:57.540528 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:57.540528 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:57.540528 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:57.540528 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:57.540528 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:20:57.540528 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:20:57.540528 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:20:57.540528 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:20:57.541505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:57.541505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:57.541505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:57.541505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:57.541505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:57.541505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:57.541505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:57.541505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:57.541505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:57.541505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:57.541505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:57.541505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:57.541505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:57.541505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:57.541505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:57.541505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:57.541505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:20:57.542546 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:20:57.542546 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:20:57.542546 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:57.544214 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 544/1068] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:57.549637 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74289ec1-365c-4752-ba1e-5138f0ff563d None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:57.613311 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:57.654477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74289ec1-365c-4752-ba1e-5138f0ff563d None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:57.688227 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72bbdc-ca99-4ac6-b0ab-b83e915b8bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:57.688838 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72bbdc-ca99-4ac6-b0ab-b83e915b8bea None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:57.689114 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72bbdc-ca99-4ac6-b0ab-b83e915b8bea None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:57.689385 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72bbdc-ca99-4ac6-b0ab-b83e915b8bea None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:57.690038 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72bbdc-ca99-4ac6-b0ab-b83e915b8bea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:57.690348 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72bbdc-ca99-4ac6-b0ab-b83e915b8bea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:57.691711 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72bbdc-ca99-4ac6-b0ab-b83e915b8bea None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:57.708472 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 525/1069] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:20:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:20:57.709763 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21661953-c767-4264-888d-22b0c6fec32b None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:57.820796 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21661953-c767-4264-888d-22b0c6fec32b None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:57.855009 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20723823-b7d0-42cb-b021-94551aafa97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:57.855631 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20723823-b7d0-42cb-b021-94551aafa97e None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:57.855906 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20723823-b7d0-42cb-b021-94551aafa97e None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:57.856174 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20723823-b7d0-42cb-b021-94551aafa97e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:57.863620 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20723823-b7d0-42cb-b021-94551aafa97e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:57.865052 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20723823-b7d0-42cb-b021-94551aafa97e None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:57.877006 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-20723823-b7d0-42cb-b021-94551aafa97e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:57.877006 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:57.877006 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:57.877006 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:57.877006 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:57.877006 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:57.877006 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:57.877006 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:57.877006 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:57.877006 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:57.877006 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:57.877006 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:57.877006 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:20:57.877006 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:20:57.877006 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:20:57.877933 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:20:57.877933 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:57.877933 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:57.877933 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:57.877933 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:57.877933 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:57.877933 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:57.877933 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:57.877933 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:57.877933 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:57.877933 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:57.877933 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:57.877933 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:57.877933 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:57.877933 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:57.877933 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:57.877933 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:57.878881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:20:57.878881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:20:57.878881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:20:57.878881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:20:57.878881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:20:57.878881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:20:57.878881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:20:57.878881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:57.879937 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 545/1070] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:20:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:57.884098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72bbdc-ca99-4ac6-b0ab-b83e915b8bea None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:57.988680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72bbdc-ca99-4ac6-b0ab-b83e915b8bea None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:58.010826 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:58.023557 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa22f3-56cf-440a-bf10-148549bfe56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:58.024155 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa22f3-56cf-440a-bf10-148549bfe56c None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:58.024433 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa22f3-56cf-440a-bf10-148549bfe56c None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:58.024702 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa22f3-56cf-440a-bf10-148549bfe56c None admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:58.025385 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aa22f3-56cf-440a-bf10-148549bfe56c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:58.025688 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aa22f3-56cf-440a-bf10-148549bfe56c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:58.027143 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aa22f3-56cf-440a-bf10-148549bfe56c None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:58.039028 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 526/1071] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:20:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:58.045321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20723823-b7d0-42cb-b021-94551aafa97e None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:58.094610 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:58.147578 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20723823-b7d0-42cb-b021-94551aafa97e None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:58.181624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ea972-f157-4ee6-8777-c324650a0a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:58.182302 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ea972-f157-4ee6-8777-c324650a0a79 None admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:58.182579 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ea972-f157-4ee6-8777-c324650a0a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:58.182848 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ea972-f157-4ee6-8777-c324650a0a79 None admin] PATH_INFO: `/v3/projects/cff3a6011f4c4893bddffdad3d69f3df/users/cee41014d5d043888c6b399710bccab4/roles/dabb7e453c8742aa809ef7543c15d1d3` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:58.185013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ea972-f157-4ee6-8777-c324650a0a79 None admin] RBAC: Authorizing `identity:create_grant(project_id=cff3a6011f4c4893bddffdad3d69f3df, user_id=cee41014d5d043888c6b399710bccab4, role_id=dabb7e453c8742aa809ef7543c15d1d3)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:58.186556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ea972-f157-4ee6-8777-c324650a0a79 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:58.202138 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 546/1072] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 10:20:58 2022] PUT /identity/v3/projects/cff3a6011f4c4893bddffdad3d69f3df/users/cee41014d5d043888c6b399710bccab4/roles/dabb7e453c8742aa809ef7543c15d1d3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:20:59.177355 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5a324-8b48-40be-b9a1-9316283016fd None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:59.177355 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5a324-8b48-40be-b9a1-9316283016fd None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:59.177355 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5a324-8b48-40be-b9a1-9316283016fd None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:59.182048 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 527/1073] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:20:59.183558 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6e08e-2490-4909-b8f9-0fa5f4255ecf None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:59.183940 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6e08e-2490-4909-b8f9-0fa5f4255ecf None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:59.184114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6e08e-2490-4909-b8f9-0fa5f4255ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:59.195910 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b27e3a-3e58-4418-bf29-3db31894884f None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:59.196187 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b27e3a-3e58-4418-bf29-3db31894884f None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:59.196456 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b27e3a-3e58-4418-bf29-3db31894884f None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:59.197869 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 528/1074] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:20:59.202538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd57a7-8408-4084-bbd6-53e51758af90 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:59.202817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd57a7-8408-4084-bbd6-53e51758af90 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:59.203085 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd57a7-8408-4084-bbd6-53e51758af90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:59.203178 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-76d6e08e-2490-4909-b8f9-0fa5f4255ecf None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:59.204300 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-76d6e08e-2490-4909-b8f9-0fa5f4255ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:59.221342 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-80cd57a7-8408-4084-bbd6-53e51758af90 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:20:59.222552 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-80cd57a7-8408-4084-bbd6-53e51758af90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:20:59.277035 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 547/1075] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:20:59.294068 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 529/1076] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:20:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 10:20:59.542964 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6e08e-2490-4909-b8f9-0fa5f4255ecf None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:59.560340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd57a7-8408-4084-bbd6-53e51758af90 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:59.587068 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:59.655816 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6e08e-2490-4909-b8f9-0fa5f4255ecf None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:59.666928 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd57a7-8408-4084-bbd6-53e51758af90 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:59.680110 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:59.695112 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e085f561-f5d2-42a5-88ca-fe3fd5db4f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:59.696206 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e085f561-f5d2-42a5-88ca-fe3fd5db4f56 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:59.696631 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e085f561-f5d2-42a5-88ca-fe3fd5db4f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:59.697048 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e085f561-f5d2-42a5-88ca-fe3fd5db4f56 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:59.701023 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2236dd-0d2f-423a-bd29-dc56d0b1a556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:59.701621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2236dd-0d2f-423a-bd29-dc56d0b1a556 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:59.701891 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2236dd-0d2f-423a-bd29-dc56d0b1a556 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:59.702183 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2236dd-0d2f-423a-bd29-dc56d0b1a556 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:59.709084 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e085f561-f5d2-42a5-88ca-fe3fd5db4f56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:20:59.714457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-e085f561-f5d2-42a5-88ca-fe3fd5db4f56 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:20:59.714457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:20:59.714457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:20:59.714457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:20:59.714457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:20:59.714457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:20:59.714457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:20:59.714457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:20:59.714457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:20:59.714457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:20:59.714457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:20:59.714457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:20:59.714457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:20:59.714457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:20:59.714457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:20:59.714457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:20:59.715566 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:20:59.715566 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:20:59.715566 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:20:59.715566 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:20:59.715566 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:20:59.715566 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:20:59.715566 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:20:59.715566 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:20:59.715566 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:20:59.715566 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:20:59.715566 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:20:59.715566 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:20:59.715566 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:20:59.715566 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:20:59.715566 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:20:59.715566 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:20:59.717107 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:20:59.717107 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:20:59.717107 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:20:59.717107 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:20:59.717107 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:20:59.717107 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:20:59.717107 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:20:59.717107 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:20:59.717107 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:20:59.717107 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:20:59.717107 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:20:59.717107 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:20:59.719156 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 548/1077] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:20:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:20:59.724976 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e085f561-f5d2-42a5-88ca-fe3fd5db4f56 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:59.772070 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2236dd-0d2f-423a-bd29-dc56d0b1a556 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:59.773557 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2236dd-0d2f-423a-bd29-dc56d0b1a556 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:59.835366 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:59.839934 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e085f561-f5d2-42a5-88ca-fe3fd5db4f56 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:20:59.878455 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f34b71-e984-4c4b-a142-f84d9b9e3b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:20:59.879272 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f34b71-e984-4c4b-a142-f84d9b9e3b95 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:20:59.879382 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 530/1078] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:20:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 26 10:20:59.879489 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f34b71-e984-4c4b-a142-f84d9b9e3b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:20:59.879676 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f34b71-e984-4c4b-a142-f84d9b9e3b95 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:20:59.880409 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f34b71-e984-4c4b-a142-f84d9b9e3b95 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:20:59.880717 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f34b71-e984-4c4b-a142-f84d9b9e3b95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:20:59.882157 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f34b71-e984-4c4b-a142-f84d9b9e3b95 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:20:59.884898 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2236dd-0d2f-423a-bd29-dc56d0b1a556 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:59.896258 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 549/1079] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:20:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:20:59.897472 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f34b71-e984-4c4b-a142-f84d9b9e3b95 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:20:59.902901 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:20:59.993715 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2236dd-0d2f-423a-bd29-dc56d0b1a556 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:00.002740 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f34b71-e984-4c4b-a142-f84d9b9e3b95 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:00.030894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6df36-69e2-4167-9760-7563113d0e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:00.031521 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6df36-69e2-4167-9760-7563113d0e93 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:00.031799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6df36-69e2-4167-9760-7563113d0e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:00.032070 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6df36-69e2-4167-9760-7563113d0e93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:00.038126 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e206854-8a24-47d6-84ff-0763a7b01a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:00.039001 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e206854-8a24-47d6-84ff-0763a7b01a84 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:00.039001 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e206854-8a24-47d6-84ff-0763a7b01a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:00.039271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e206854-8a24-47d6-84ff-0763a7b01a84 None admin] PATH_INFO: `/v3/users/25376c83a3934017873ce7cde98bf027` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:00.040386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e206854-8a24-47d6-84ff-0763a7b01a84 None admin] RBAC: Authorizing `identity:get_user(user_id=25376c83a3934017873ce7cde98bf027)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:00.041799 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e206854-8a24-47d6-84ff-0763a7b01a84 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:00.042966 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b6df36-69e2-4167-9760-7563113d0e93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:00.043590 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 550/1080] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:20:59 2022] GET /identity/v3/users/25376c83a3934017873ce7cde98bf027 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:21:00.044644 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b6df36-69e2-4167-9760-7563113d0e93 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:00.049047 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e206854-8a24-47d6-84ff-0763a7b01a84 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:00.059959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-64b6df36-69e2-4167-9760-7563113d0e93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:21:00.059959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:21:00.059959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:21:00.059959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:21:00.059959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:21:00.059959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:21:00.059959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:21:00.059959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:21:00.059959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:21:00.059959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:21:00.059959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:21:00.059959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:21:00.059959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:21:00.059959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:21:00.059959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:21:00.059959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:21:00.061098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:21:00.061098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:21:00.061098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:21:00.061098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:21:00.061098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:21:00.061098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:21:00.061098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:21:00.061098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:21:00.061098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:21:00.061098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:21:00.061098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:21:00.061098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:21:00.061098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:21:00.061098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:21:00.061098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:21:00.061098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:21:00.061098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:21:00.062012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:21:00.062012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:21:00.062012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:21:00.062012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 531/1081] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:20:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:21:00.066736 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6df36-69e2-4167-9760-7563113d0e93 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:00.084204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:00.130397 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:00.160718 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e206854-8a24-47d6-84ff-0763a7b01a84 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:00.170745 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6df36-69e2-4167-9760-7563113d0e93 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:00.195279 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb722c5a-5a83-4114-8b86-f35bbdc973b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:00.195908 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb722c5a-5a83-4114-8b86-f35bbdc973b4 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:00.196205 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb722c5a-5a83-4114-8b86-f35bbdc973b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:00.196476 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb722c5a-5a83-4114-8b86-f35bbdc973b4 None admin] PATH_INFO: `/v3/projects/4df4efbd6e0440b490af925e22b13188` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:00.197576 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb722c5a-5a83-4114-8b86-f35bbdc973b4 None admin] RBAC: Authorizing `identity:get_project(project_id=4df4efbd6e0440b490af925e22b13188)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:00.199020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb722c5a-5a83-4114-8b86-f35bbdc973b4 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:00.201333 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 551/1082] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:21:00 2022] GET /identity/v3/projects/4df4efbd6e0440b490af925e22b13188 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:21:00.206951 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb722c5a-5a83-4114-8b86-f35bbdc973b4 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:00.207350 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06057df4-3fc5-4da8-ac03-080b2a6ecf70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:00.208210 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06057df4-3fc5-4da8-ac03-080b2a6ecf70 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:00.208488 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06057df4-3fc5-4da8-ac03-080b2a6ecf70 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:00.208760 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06057df4-3fc5-4da8-ac03-080b2a6ecf70 None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:00.209405 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06057df4-3fc5-4da8-ac03-080b2a6ecf70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:21:00.209707 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06057df4-3fc5-4da8-ac03-080b2a6ecf70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:00.211105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06057df4-3fc5-4da8-ac03-080b2a6ecf70 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:00.226312 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 532/1083] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:21:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:00.325136 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:00.332250 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb722c5a-5a83-4114-8b86-f35bbdc973b4 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:00.369556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce9adf-337c-43bb-a8f5-1b621f81d7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:00.370303 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce9adf-337c-43bb-a8f5-1b621f81d7f1 None admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:00.370540 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce9adf-337c-43bb-a8f5-1b621f81d7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:00.370818 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce9adf-337c-43bb-a8f5-1b621f81d7f1 None admin] PATH_INFO: `/v3/projects/4df4efbd6e0440b490af925e22b13188/users/25376c83a3934017873ce7cde98bf027/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:00.372849 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec541b0a-a1d7-489f-b857-26e8b1747018 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:00.373135 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec541b0a-a1d7-489f-b857-26e8b1747018 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:00.373409 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec541b0a-a1d7-489f-b857-26e8b1747018 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:00.373517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce9adf-337c-43bb-a8f5-1b621f81d7f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=4df4efbd6e0440b490af925e22b13188, user_id=25376c83a3934017873ce7cde98bf027, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:00.374973 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 533/1084] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:21:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:21:00.375084 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce9adf-337c-43bb-a8f5-1b621f81d7f1 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:00.380807 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493a5a8-5df6-4b61-95b7-6b03f6b1ecda None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:00.381098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493a5a8-5df6-4b61-95b7-6b03f6b1ecda None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:00.381461 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493a5a8-5df6-4b61-95b7-6b03f6b1ecda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:00.394124 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 552/1085] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 10:21:00 2022] PUT /identity/v3/projects/4df4efbd6e0440b490af925e22b13188/users/25376c83a3934017873ce7cde98bf027/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:21:00.401840 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-c493a5a8-5df6-4b61-95b7-6b03f6b1ecda None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:21:00.402829 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-c493a5a8-5df6-4b61-95b7-6b03f6b1ecda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:21:00.484351 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 534/1086] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:21:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 10:21:00.759782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce9adf-337c-43bb-a8f5-1b621f81d7f1 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:00.863265 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce9adf-337c-43bb-a8f5-1b621f81d7f1 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:00.897808 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e20df-d2f8-495a-baaf-52bc3cebf8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:00.898457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e20df-d2f8-495a-baaf-52bc3cebf8bf None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:00.898735 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e20df-d2f8-495a-baaf-52bc3cebf8bf None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:00.899007 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e20df-d2f8-495a-baaf-52bc3cebf8bf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:00.909868 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2e20df-d2f8-495a-baaf-52bc3cebf8bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:21:00.914630 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-5c2e20df-d2f8-495a-baaf-52bc3cebf8bf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 10:21:00.914630 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:21:00.914630 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:21:00.914630 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:21:00.914630 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:21:00.914630 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:21:00.914630 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:21:00.914630 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:21:00.914630 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:21:00.914630 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:21:00.914630 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:21:00.914630 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:21:00.914630 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:21:00.914630 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:21:00.914630 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:21:00.914630 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:21:00.915731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:21:00.915731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:21:00.915731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:21:00.915731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:21:00.915731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:21:00.915731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:21:00.915731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:21:00.915731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:21:00.915731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:21:00.915731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:21:00.915731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:21:00.915731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:21:00.915731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:21:00.915731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:21:00.915731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:21:00.915731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:21:00.916617 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:21:00.916617 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:21:00.916617 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:21:00.916617 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:21:00.916617 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:21:00.916617 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:21:00.916617 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:21:00.916617 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:21:00.916617 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:21:00.916617 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:21:00.916617 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 10:21:00.916617 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:21:00.916617 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 553/1087] 10.222.0.49 () {56 vars in 1120 bytes} [Mon Sep 26 10:21:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:21:00.921808 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493a5a8-5df6-4b61-95b7-6b03f6b1ecda None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:00.975060 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:01.018100 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493a5a8-5df6-4b61-95b7-6b03f6b1ecda None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:01.051319 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ec812-d93f-4fce-966e-41d42d83832d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:01.051926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ec812-d93f-4fce-966e-41d42d83832d None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:01.052204 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ec812-d93f-4fce-966e-41d42d83832d None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:01.052474 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ec812-d93f-4fce-966e-41d42d83832d None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:01.053126 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ec812-d93f-4fce-966e-41d42d83832d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:21:01.053425 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ec812-d93f-4fce-966e-41d42d83832d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:01.054827 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ec812-d93f-4fce-966e-41d42d83832d None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:01.062716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 535/1088] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Sep 26 10:21:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:01.068658 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e20df-d2f8-495a-baaf-52bc3cebf8bf None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:01.087491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:01.177716 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e20df-d2f8-495a-baaf-52bc3cebf8bf None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:01.211676 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f3a64-836a-4011-b47a-b8591c533349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:01.212300 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f3a64-836a-4011-b47a-b8591c533349 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:01.212581 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f3a64-836a-4011-b47a-b8591c533349 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:01.212852 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f3a64-836a-4011-b47a-b8591c533349 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:01.223162 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f3a64-836a-4011-b47a-b8591c533349 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:01.224606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f3a64-836a-4011-b47a-b8591c533349 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:01.238473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-576f3a64-836a-4011-b47a-b8591c533349 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:21:01.238473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:21:01.238473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:21:01.238473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:21:01.238473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:21:01.238473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:21:01.238473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:21:01.238473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:21:01.238473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:21:01.238473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:21:01.238473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:21:01.238473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:21:01.238473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:21:01.238473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:21:01.238473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:21:01.238473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:21:01.239441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:21:01.239441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:21:01.239441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:21:01.239441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:21:01.239441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:21:01.239441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:21:01.239441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:21:01.239441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:21:01.239441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:21:01.239441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:21:01.239441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:21:01.239441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:21:01.239441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:21:01.239441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:21:01.239441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:21:01.239441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:21:01.239441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:21:01.240341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:21:01.240341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:21:01.240341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:21:01.240341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 554/1089] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:21:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:21:01.245463 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ec812-d93f-4fce-966e-41d42d83832d None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:01.344510 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ec812-d93f-4fce-966e-41d42d83832d None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:01.359799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:01.377912 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e423d26-88a6-491b-86ce-b4a174f400dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:01.378538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e423d26-88a6-491b-86ce-b4a174f400dd None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:01.378810 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e423d26-88a6-491b-86ce-b4a174f400dd None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:01.379076 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e423d26-88a6-491b-86ce-b4a174f400dd None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:01.379708 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e423d26-88a6-491b-86ce-b4a174f400dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:21:01.380002 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e423d26-88a6-491b-86ce-b4a174f400dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:01.381340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e423d26-88a6-491b-86ce-b4a174f400dd None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:01.393127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 536/1090] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:21:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:01.398768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f3a64-836a-4011-b47a-b8591c533349 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:01.464525 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:01.497515 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f3a64-836a-4011-b47a-b8591c533349 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:01.532318 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84443b54-67ee-4fa0-b6bb-ae40f0195d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:01.532941 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84443b54-67ee-4fa0-b6bb-ae40f0195d85 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:01.533214 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84443b54-67ee-4fa0-b6bb-ae40f0195d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:01.533482 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84443b54-67ee-4fa0-b6bb-ae40f0195d85 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:01.541232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84443b54-67ee-4fa0-b6bb-ae40f0195d85 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:01.542694 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84443b54-67ee-4fa0-b6bb-ae40f0195d85 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:01.553961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-84443b54-67ee-4fa0-b6bb-ae40f0195d85 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 10:21:01.553961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:21:01.553961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:21:01.553961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:21:01.553961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:21:01.553961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:21:01.553961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:21:01.553961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:21:01.553961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:21:01.553961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:21:01.553961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:21:01.553961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:21:01.553961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:21:01.553961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:21:01.553961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:21:01.553961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:21:01.555116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:21:01.555116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:21:01.555116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:21:01.555116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:21:01.555116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:21:01.555116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:21:01.555116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:21:01.555116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:21:01.555116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:21:01.555116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:21:01.555116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:21:01.555116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:21:01.555116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:21:01.555116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:21:01.555116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:21:01.555116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:21:01.555116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:21:01.555116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:21:01.556379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:21:01.556379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:21:01.556379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:21:01.556379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:21:01.556379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:21:01.556379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:21:01.556379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:21:01.556379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:21:01.556379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 10:21:01.556379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:21:01.556379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 555/1091] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Sep 26 10:21:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:21:01.561483 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e423d26-88a6-491b-86ce-b4a174f400dd None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:01.658403 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e423d26-88a6-491b-86ce-b4a174f400dd None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:01.691781 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e8177f-5b9c-46fe-80a8-e1220380fb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:01.692371 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8177f-5b9c-46fe-80a8-e1220380fb85 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:01.692643 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8177f-5b9c-46fe-80a8-e1220380fb85 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:01.692910 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8177f-5b9c-46fe-80a8-e1220380fb85 None admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:01.693622 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e8177f-5b9c-46fe-80a8-e1220380fb85 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:21:01.693920 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e8177f-5b9c-46fe-80a8-e1220380fb85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:01.695352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e8177f-5b9c-46fe-80a8-e1220380fb85 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:01.711868 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 537/1092] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Sep 26 10:21:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:01.717968 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84443b54-67ee-4fa0-b6bb-ae40f0195d85 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:01.818942 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84443b54-67ee-4fa0-b6bb-ae40f0195d85 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:01.844362 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:01.852442 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88857d86-dbda-4269-a561-d0507c4773bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:01.853045 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88857d86-dbda-4269-a561-d0507c4773bf None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:01.853325 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88857d86-dbda-4269-a561-d0507c4773bf None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:01.853601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88857d86-dbda-4269-a561-d0507c4773bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:01.920486 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88857d86-dbda-4269-a561-d0507c4773bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:01.921844 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88857d86-dbda-4269-a561-d0507c4773bf None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:02.017115 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 556/1093] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:21:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 10:21:02.022991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e8177f-5b9c-46fe-80a8-e1220380fb85 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:02.056568 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:02.121088 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e8177f-5b9c-46fe-80a8-e1220380fb85 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:02.154007 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a33f5d6-5de6-4163-9a56-787f4c4eca57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:02.154650 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33f5d6-5de6-4163-9a56-787f4c4eca57 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:02.154923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33f5d6-5de6-4163-9a56-787f4c4eca57 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:02.155190 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33f5d6-5de6-4163-9a56-787f4c4eca57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:02.165069 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a33f5d6-5de6-4163-9a56-787f4c4eca57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:02.166523 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a33f5d6-5de6-4163-9a56-787f4c4eca57 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:02.182050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-7a33f5d6-5de6-4163-9a56-787f4c4eca57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:21:02.182050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:21:02.182050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:21:02.182050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:21:02.182050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:21:02.182050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:21:02.182050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:21:02.182050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:21:02.182050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:21:02.182050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:21:02.182050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:21:02.182050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:21:02.182050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:21:02.182050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:21:02.182050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:21:02.182050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:21:02.188149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:21:02.188149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:21:02.188149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:21:02.188149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:21:02.188149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:21:02.188149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:21:02.188149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:21:02.188149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:21:02.188149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:21:02.188149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:21:02.188149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:21:02.188149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:21:02.188149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:21:02.188149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:21:02.188149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:21:02.188149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:21:02.188149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:21:02.189293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:21:02.189293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:21:02.189293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:21:02.189293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 538/1094] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:21:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:21:02.189585 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88857d86-dbda-4269-a561-d0507c4773bf None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:02.215562 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:02.291644 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88857d86-dbda-4269-a561-d0507c4773bf None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:02.327759 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300ed94-b9ec-4378-aaa0-81677c4e2f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:02.328358 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300ed94-b9ec-4378-aaa0-81677c4e2f90 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:02.328639 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300ed94-b9ec-4378-aaa0-81677c4e2f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:02.328910 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300ed94-b9ec-4378-aaa0-81677c4e2f90 None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:02.329547 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300ed94-b9ec-4378-aaa0-81677c4e2f90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:21:02.329849 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300ed94-b9ec-4378-aaa0-81677c4e2f90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:02.331238 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300ed94-b9ec-4378-aaa0-81677c4e2f90 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:02.343223 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 557/1095] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:21:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:02.349060 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a33f5d6-5de6-4163-9a56-787f4c4eca57 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:02.437342 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:02.453903 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a33f5d6-5de6-4163-9a56-787f4c4eca57 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:02.487662 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa566da0-ee42-471e-9c2c-c700e5a37536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:02.488279 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa566da0-ee42-471e-9c2c-c700e5a37536 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:02.488553 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa566da0-ee42-471e-9c2c-c700e5a37536 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:02.488817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa566da0-ee42-471e-9c2c-c700e5a37536 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:02.495363 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa566da0-ee42-471e-9c2c-c700e5a37536 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:02.496788 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa566da0-ee42-471e-9c2c-c700e5a37536 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:02.507668 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-aa566da0-ee42-471e-9c2c-c700e5a37536 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:21:02.507668 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:21:02.507668 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:21:02.507668 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:21:02.507668 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:21:02.507668 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:21:02.507668 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:21:02.507668 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:21:02.507668 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:21:02.507668 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:21:02.507668 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:21:02.507668 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:21:02.507668 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:21:02.507668 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:21:02.507668 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:21:02.508566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:21:02.508566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:21:02.508566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:21:02.508566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:21:02.508566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:21:02.508566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:21:02.508566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:21:02.508566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:21:02.508566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:21:02.508566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:21:02.508566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:21:02.508566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:21:02.508566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:21:02.508566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:21:02.508566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:21:02.508566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:21:02.508566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:21:02.509551 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:21:02.509551 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:21:02.509551 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:21:02.509551 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:21:02.509551 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:21:02.509551 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:21:02.509551 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:21:02.509551 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:21:02.509551 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 539/1096] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:21:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:21:02.515232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300ed94-b9ec-4378-aaa0-81677c4e2f90 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:02.517283 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4adcabd-3e79-40a6-818f-ebee61dc31b6 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:02.517567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4adcabd-3e79-40a6-818f-ebee61dc31b6 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:02.517844 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4adcabd-3e79-40a6-818f-ebee61dc31b6 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:02.519152 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 540/1097] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:21:02.524051 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aac25c-113d-4d81-aa0c-a7cf61c57919 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:02.524327 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aac25c-113d-4d81-aa0c-a7cf61c57919 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:02.524592 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aac25c-113d-4d81-aa0c-a7cf61c57919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:02.542722 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-80aac25c-113d-4d81-aa0c-a7cf61c57919 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:21:02.543931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-80aac25c-113d-4d81-aa0c-a7cf61c57919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:21:02.604928 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:02.610787 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 541/1098] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:21:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 10:21:02.616610 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300ed94-b9ec-4378-aaa0-81677c4e2f90 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:02.649524 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9db623-a264-49c5-93a4-219fba850445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:02.650136 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9db623-a264-49c5-93a4-219fba850445 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:02.650411 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9db623-a264-49c5-93a4-219fba850445 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:02.650678 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9db623-a264-49c5-93a4-219fba850445 None admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:02.651381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9db623-a264-49c5-93a4-219fba850445 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:21:02.651683 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9db623-a264-49c5-93a4-219fba850445 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:02.653080 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9db623-a264-49c5-93a4-219fba850445 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:02.664729 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 558/1099] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:21:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:02.670793 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aac25c-113d-4d81-aa0c-a7cf61c57919 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:02.745763 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:02.768181 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aac25c-113d-4d81-aa0c-a7cf61c57919 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:02.801801 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4e107-76d7-44cd-995a-5df294193097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:02.802443 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4e107-76d7-44cd-995a-5df294193097 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:02.802723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4e107-76d7-44cd-995a-5df294193097 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:02.802996 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4e107-76d7-44cd-995a-5df294193097 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:02.803720 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e4e107-76d7-44cd-995a-5df294193097 None admin] RBAC: Adding query filter params (role.id=dabb7e453c8742aa809ef7543c15d1d3, scope.project.id=cff3a6011f4c4893bddffdad3d69f3df, user.id=cee41014d5d043888c6b399710bccab4) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:21:02.804505 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e4e107-76d7-44cd-995a-5df294193097 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:02.805436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e4e107-76d7-44cd-995a-5df294193097 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:02.811183 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 542/1100] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:21:02 2022] GET /identity/v3/role_assignments?user.id=cee41014d5d043888c6b399710bccab4&scope.project.id=cff3a6011f4c4893bddffdad3d69f3df&role.id=dabb7e453c8742aa809ef7543c15d1d3 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:02.878605 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9db623-a264-49c5-93a4-219fba850445 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:02.991724 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9db623-a264-49c5-93a4-219fba850445 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:03.027231 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02351b-b65b-4f00-94ca-24f67e8013a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:03.027862 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02351b-b65b-4f00-94ca-24f67e8013a9 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:03.028150 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02351b-b65b-4f00-94ca-24f67e8013a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:03.029033 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02351b-b65b-4f00-94ca-24f67e8013a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:03.039236 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c02351b-b65b-4f00-94ca-24f67e8013a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:21:03.044081 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-4c02351b-b65b-4f00-94ca-24f67e8013a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:21:03.044081 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:21:03.044081 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:21:03.044081 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:21:03.044081 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:21:03.044081 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:21:03.044081 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:21:03.044081 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:21:03.044081 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:21:03.044081 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:21:03.044081 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:21:03.044081 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:21:03.044081 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:21:03.044081 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:21:03.044081 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:21:03.044081 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:21:03.045011 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:21:03.045011 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:21:03.045011 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:21:03.045011 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:21:03.045011 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:21:03.045011 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:21:03.045011 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:21:03.045011 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:21:03.045011 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:21:03.045011 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:21:03.045011 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:21:03.045011 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:21:03.045011 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:21:03.045011 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:21:03.045011 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:21:03.045011 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:21:03.045873 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:21:03.045873 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:21:03.045873 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:21:03.045873 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:21:03.045873 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:21:03.045873 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:21:03.045873 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:21:03.045873 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:21:03.045873 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:21:03.045873 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:21:03.045873 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:21:03.045873 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:21:03.045873 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 559/1101] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:21:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:21:03.051381 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4e107-76d7-44cd-995a-5df294193097 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:03.153951 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4e107-76d7-44cd-995a-5df294193097 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:03.187189 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06184b6d-922f-4992-8474-37f492c36c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:03.187795 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06184b6d-922f-4992-8474-37f492c36c97 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:03.188064 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06184b6d-922f-4992-8474-37f492c36c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:03.188331 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06184b6d-922f-4992-8474-37f492c36c97 None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:03.188987 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06184b6d-922f-4992-8474-37f492c36c97 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:21:03.189286 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06184b6d-922f-4992-8474-37f492c36c97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:03.190666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06184b6d-922f-4992-8474-37f492c36c97 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:03.199268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 543/1102] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:21:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:03.205833 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02351b-b65b-4f00-94ca-24f67e8013a9 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:03.222676 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:03.319431 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02351b-b65b-4f00-94ca-24f67e8013a9 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:03.354052 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad4609-4ab6-43b5-95af-145febb37c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:03.354672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad4609-4ab6-43b5-95af-145febb37c5f None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:03.354947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad4609-4ab6-43b5-95af-145febb37c5f None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:03.355214 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad4609-4ab6-43b5-95af-145febb37c5f None admin] PATH_INFO: `/v3/users/25376c83a3934017873ce7cde98bf027` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:03.356382 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad4609-4ab6-43b5-95af-145febb37c5f None admin] RBAC: Authorizing `identity:get_user(user_id=25376c83a3934017873ce7cde98bf027)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:03.357807 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad4609-4ab6-43b5-95af-145febb37c5f None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:03.359603 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 560/1103] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:21:03 2022] GET /identity/v3/users/25376c83a3934017873ce7cde98bf027 => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:21:03.365285 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06184b6d-922f-4992-8474-37f492c36c97 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:03.368127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:03.471795 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06184b6d-922f-4992-8474-37f492c36c97 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:03.506789 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439ee39-69e9-467a-a337-c34a202c0acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:03.507384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439ee39-69e9-467a-a337-c34a202c0acc None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:03.507658 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439ee39-69e9-467a-a337-c34a202c0acc None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:03.507924 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1439ee39-69e9-467a-a337-c34a202c0acc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:03.576987 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1439ee39-69e9-467a-a337-c34a202c0acc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:03.578698 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1439ee39-69e9-467a-a337-c34a202c0acc None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:03.580824 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:03.677430 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 544/1104] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:21:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 10:21:03.683388 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad4609-4ab6-43b5-95af-145febb37c5f None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:03.758535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:03.785957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad4609-4ab6-43b5-95af-145febb37c5f None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:03.819815 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b1806b-f1fc-4bae-b407-81d20a2724af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:03.820431 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1806b-f1fc-4bae-b407-81d20a2724af None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:03.820703 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1806b-f1fc-4bae-b407-81d20a2724af None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:03.820977 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1806b-f1fc-4bae-b407-81d20a2724af None admin] PATH_INFO: `/v3/projects/4df4efbd6e0440b490af925e22b13188` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:03.822089 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b1806b-f1fc-4bae-b407-81d20a2724af None admin] RBAC: Authorizing `identity:get_project(project_id=4df4efbd6e0440b490af925e22b13188)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:03.823497 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b1806b-f1fc-4bae-b407-81d20a2724af None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:03.825311 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 561/1105] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:21:03 2022] GET /identity/v3/projects/4df4efbd6e0440b490af925e22b13188 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:03.831211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439ee39-69e9-467a-a337-c34a202c0acc None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:03.927755 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1439ee39-69e9-467a-a337-c34a202c0acc None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:03.929248 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:03.962463 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f446e0-927b-4b9e-9cf3-96856fb30ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:03.963051 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f446e0-927b-4b9e-9cf3-96856fb30ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:03.963328 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f446e0-927b-4b9e-9cf3-96856fb30ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:03.963599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f446e0-927b-4b9e-9cf3-96856fb30ac7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:03.964323 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f446e0-927b-4b9e-9cf3-96856fb30ac7 None admin] RBAC: Adding query filter params (role.id=532b37d9076c45618cf352aafb85f35e, scope.project.id=4df4efbd6e0440b490af925e22b13188, user.id=25376c83a3934017873ce7cde98bf027) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:21:03.964628 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f446e0-927b-4b9e-9cf3-96856fb30ac7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:03.966113 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f446e0-927b-4b9e-9cf3-96856fb30ac7 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:03.971695 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 545/1106] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:21:03 2022] GET /identity/v3/role_assignments?user.id=25376c83a3934017873ce7cde98bf027&scope.project.id=4df4efbd6e0440b490af925e22b13188&role.id=532b37d9076c45618cf352aafb85f35e => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:06.101209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4cd25-1702-4654-abc4-f14126fff2a4 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:06.101758 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4cd25-1702-4654-abc4-f14126fff2a4 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:06.101758 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4cd25-1702-4654-abc4-f14126fff2a4 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:06.103029 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 562/1107] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:21:06.108762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ceb24-9560-49a3-a088-7c9cdb1bc89f None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:06.109153 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ceb24-9560-49a3-a088-7c9cdb1bc89f None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:06.109310 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ceb24-9560-49a3-a088-7c9cdb1bc89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:06.127906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-f06ceb24-9560-49a3-a088-7c9cdb1bc89f None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:21:06.129094 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-f06ceb24-9560-49a3-a088-7c9cdb1bc89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:21:06.192942 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 546/1108] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:21:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:21:06.462776 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4cd25-1702-4654-abc4-f14126fff2a4 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:06.560782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4cd25-1702-4654-abc4-f14126fff2a4 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:06.593121 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f1282f-ee0f-4d4e-b624-f192c476c378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:06.593720 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1282f-ee0f-4d4e-b624-f192c476c378 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:06.593994 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1282f-ee0f-4d4e-b624-f192c476c378 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:06.594293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1282f-ee0f-4d4e-b624-f192c476c378 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:06.595374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f1282f-ee0f-4d4e-b624-f192c476c378 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:06.596770 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f1282f-ee0f-4d4e-b624-f192c476c378 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:06.598312 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 563/1109] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:21:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:06.603858 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ceb24-9560-49a3-a088-7c9cdb1bc89f None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:06.695109 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:06.701428 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ceb24-9560-49a3-a088-7c9cdb1bc89f None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:06.735430 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a380a-65db-4e3a-b2e8-ee867129ca05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:06.735967 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a380a-65db-4e3a-b2e8-ee867129ca05 None admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:06.736243 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a380a-65db-4e3a-b2e8-ee867129ca05 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:06.736540 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a380a-65db-4e3a-b2e8-ee867129ca05 None admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:06.737176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a380a-65db-4e3a-b2e8-ee867129ca05 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:06.738625 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a380a-65db-4e3a-b2e8-ee867129ca05 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:06.758069 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 547/1110] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 10:21:06 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:21:08.910043 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17bbcef-6a18-4200-8f25-908b7dfb3997 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:08.910583 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17bbcef-6a18-4200-8f25-908b7dfb3997 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:08.910583 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17bbcef-6a18-4200-8f25-908b7dfb3997 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:08.911983 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 564/1111] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:21:08.917067 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38144600-fb13-4727-bf3c-867c0ad82cb5 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:08.917511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38144600-fb13-4727-bf3c-867c0ad82cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:08.917613 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38144600-fb13-4727-bf3c-867c0ad82cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:08.935945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-38144600-fb13-4727-bf3c-867c0ad82cb5 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:21:08.937121 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-38144600-fb13-4727-bf3c-867c0ad82cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:21:09.003264 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 548/1112] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:21:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 10:21:09.273793 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17bbcef-6a18-4200-8f25-908b7dfb3997 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:09.288954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:09.373585 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17bbcef-6a18-4200-8f25-908b7dfb3997 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:09.406123 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e74b8-3a4e-4341-b73d-9627653ae6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:09.406732 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e74b8-3a4e-4341-b73d-9627653ae6a7 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:09.407009 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e74b8-3a4e-4341-b73d-9627653ae6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:09.407280 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e74b8-3a4e-4341-b73d-9627653ae6a7 None admin] PATH_INFO: `/v3/roles/d1109db57b004dd9a4f84a634ff76205` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:09.408649 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e74b8-3a4e-4341-b73d-9627653ae6a7 None admin] RBAC: Authorizing `identity:get_role(role_id=d1109db57b004dd9a4f84a634ff76205)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:09.410079 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e74b8-3a4e-4341-b73d-9627653ae6a7 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:09.414054 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 565/1113] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:21:09 2022] GET /identity/v3/roles/d1109db57b004dd9a4f84a634ff76205 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:09.417915 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38144600-fb13-4727-bf3c-867c0ad82cb5 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:09.498183 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:09.516987 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38144600-fb13-4727-bf3c-867c0ad82cb5 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:09.549262 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd2ef0-3c20-466d-a130-9f126ccf4758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:09.549825 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd2ef0-3c20-466d-a130-9f126ccf4758 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:09.550131 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd2ef0-3c20-466d-a130-9f126ccf4758 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:09.550402 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd2ef0-3c20-466d-a130-9f126ccf4758 None admin] PATH_INFO: `/v3/users/42a7715a8c594a70969c2af1cf6eb188` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:09.563399 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd2ef0-3c20-466d-a130-9f126ccf4758 None admin] RBAC: Authorizing `identity:get_user(user_id=42a7715a8c594a70969c2af1cf6eb188)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:09.564837 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd2ef0-3c20-466d-a130-9f126ccf4758 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:09.566447 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 549/1114] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:21:09 2022] GET /identity/v3/users/42a7715a8c594a70969c2af1cf6eb188 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:09.572160 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e74b8-3a4e-4341-b73d-9627653ae6a7 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:09.650368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:09.668898 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e74b8-3a4e-4341-b73d-9627653ae6a7 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:09.701572 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1ec0f-3033-47ad-817e-dd6ab263bab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:09.702182 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1ec0f-3033-47ad-817e-dd6ab263bab8 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:09.702457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1ec0f-3033-47ad-817e-dd6ab263bab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:09.702723 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1ec0f-3033-47ad-817e-dd6ab263bab8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:09.766554 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1ec0f-3033-47ad-817e-dd6ab263bab8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:09.767964 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1ec0f-3033-47ad-817e-dd6ab263bab8 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:09.843765 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:09.863382 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 566/1115] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:21:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 10:21:09.869260 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd2ef0-3c20-466d-a130-9f126ccf4758 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:09.965284 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd2ef0-3c20-466d-a130-9f126ccf4758 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:09.999042 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e912f-9758-455b-a572-dbad87af8035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:09.999664 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e912f-9758-455b-a572-dbad87af8035 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:09.999940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e912f-9758-455b-a572-dbad87af8035 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:10.000237 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e912f-9758-455b-a572-dbad87af8035 None admin] PATH_INFO: `/v3/projects/4df4efbd6e0440b490af925e22b13188` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:10.001335 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841e912f-9758-455b-a572-dbad87af8035 None admin] RBAC: Authorizing `identity:get_project(project_id=4df4efbd6e0440b490af925e22b13188)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:10.002777 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841e912f-9758-455b-a572-dbad87af8035 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:10.004368 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 550/1116] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:21:09 2022] GET /identity/v3/projects/4df4efbd6e0440b490af925e22b13188 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:10.010406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1ec0f-3033-47ad-817e-dd6ab263bab8 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:10.107354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1ec0f-3033-47ad-817e-dd6ab263bab8 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:10.139895 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dddd612-58e1-49d0-a72c-6416f008fcba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:10.140480 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddd612-58e1-49d0-a72c-6416f008fcba None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:10.140758 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddd612-58e1-49d0-a72c-6416f008fcba None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:10.141026 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddd612-58e1-49d0-a72c-6416f008fcba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:10.141735 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dddd612-58e1-49d0-a72c-6416f008fcba None admin] RBAC: Adding query filter params (role.id=d1109db57b004dd9a4f84a634ff76205, scope.project.id=4df4efbd6e0440b490af925e22b13188, user.id=42a7715a8c594a70969c2af1cf6eb188) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:21:10.142060 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dddd612-58e1-49d0-a72c-6416f008fcba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:10.143461 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dddd612-58e1-49d0-a72c-6416f008fcba None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:10.148581 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 567/1117] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:21:10 2022] GET /identity/v3/role_assignments?user.id=42a7715a8c594a70969c2af1cf6eb188&scope.project.id=4df4efbd6e0440b490af925e22b13188&role.id=d1109db57b004dd9a4f84a634ff76205 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:12.230378 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1550c-0c3a-42e9-aabc-3e6df037c3c3 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:12.230885 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1550c-0c3a-42e9-aabc-3e6df037c3c3 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:12.230885 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1550c-0c3a-42e9-aabc-3e6df037c3c3 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:12.232189 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 551/1118] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:21:12.237163 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5434c-3eb3-4735-9e2f-83273a031f77 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:12.237548 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5434c-3eb3-4735-9e2f-83273a031f77 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:12.237702 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5434c-3eb3-4735-9e2f-83273a031f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:12.255259 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-1ce5434c-3eb3-4735-9e2f-83273a031f77 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:21:12.256428 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-1ce5434c-3eb3-4735-9e2f-83273a031f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:21:12.297361 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:12.325641 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 568/1119] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:21:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:21:12.598953 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1550c-0c3a-42e9-aabc-3e6df037c3c3 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:12.609755 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:12.698768 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1550c-0c3a-42e9-aabc-3e6df037c3c3 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:12.731943 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d04592-e2b4-4b9a-9d69-479bc2dcdf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:12.732555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d04592-e2b4-4b9a-9d69-479bc2dcdf88 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:12.732832 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d04592-e2b4-4b9a-9d69-479bc2dcdf88 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:12.733102 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d04592-e2b4-4b9a-9d69-479bc2dcdf88 None admin] PATH_INFO: `/v3/roles/d1109db57b004dd9a4f84a634ff76205` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:12.734501 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d04592-e2b4-4b9a-9d69-479bc2dcdf88 None admin] RBAC: Authorizing `identity:get_role(role_id=d1109db57b004dd9a4f84a634ff76205)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:12.735861 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d04592-e2b4-4b9a-9d69-479bc2dcdf88 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:12.737143 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 552/1120] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:21:12 2022] GET /identity/v3/roles/d1109db57b004dd9a4f84a634ff76205 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:12.742874 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5434c-3eb3-4735-9e2f-83273a031f77 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:12.838672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5434c-3eb3-4735-9e2f-83273a031f77 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:12.871116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934420d9-7bed-4fa7-ac06-78882ab793d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:12.871679 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934420d9-7bed-4fa7-ac06-78882ab793d7 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:12.872533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934420d9-7bed-4fa7-ac06-78882ab793d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:12.872812 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934420d9-7bed-4fa7-ac06-78882ab793d7 None admin] PATH_INFO: `/v3/users/42a7715a8c594a70969c2af1cf6eb188` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:12.873918 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934420d9-7bed-4fa7-ac06-78882ab793d7 None admin] RBAC: Authorizing `identity:get_user(user_id=42a7715a8c594a70969c2af1cf6eb188)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:12.875368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934420d9-7bed-4fa7-ac06-78882ab793d7 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:12.876930 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 569/1121] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:21:12 2022] GET /identity/v3/users/42a7715a8c594a70969c2af1cf6eb188 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:12.882589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d04592-e2b4-4b9a-9d69-479bc2dcdf88 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:12.955808 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:12.979730 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d04592-e2b4-4b9a-9d69-479bc2dcdf88 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:13.014758 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db6be3-72b4-420b-9cfb-2bf99ff4b35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:13.015379 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db6be3-72b4-420b-9cfb-2bf99ff4b35f None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:13.015657 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db6be3-72b4-420b-9cfb-2bf99ff4b35f None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:13.015929 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db6be3-72b4-420b-9cfb-2bf99ff4b35f None admin] PATH_INFO: `/v3/projects/4df4efbd6e0440b490af925e22b13188` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:13.017041 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64db6be3-72b4-420b-9cfb-2bf99ff4b35f None admin] RBAC: Authorizing `identity:get_project(project_id=4df4efbd6e0440b490af925e22b13188)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:13.018507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64db6be3-72b4-420b-9cfb-2bf99ff4b35f None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:13.020115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 553/1122] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:21:12 2022] GET /identity/v3/projects/4df4efbd6e0440b490af925e22b13188 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:13.025930 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934420d9-7bed-4fa7-ac06-78882ab793d7 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:13.068278 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:13.123124 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934420d9-7bed-4fa7-ac06-78882ab793d7 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:13.154754 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c552f073-7ca7-4d8f-8f2a-91576e1ad92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:13.155385 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552f073-7ca7-4d8f-8f2a-91576e1ad92f None admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:13.155660 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552f073-7ca7-4d8f-8f2a-91576e1ad92f None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:13.155929 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552f073-7ca7-4d8f-8f2a-91576e1ad92f None admin] PATH_INFO: `/v3/projects/4df4efbd6e0440b490af925e22b13188/users/42a7715a8c594a70969c2af1cf6eb188/roles/d1109db57b004dd9a4f84a634ff76205` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:13.158042 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c552f073-7ca7-4d8f-8f2a-91576e1ad92f None admin] RBAC: Authorizing `identity:create_grant(project_id=4df4efbd6e0440b490af925e22b13188, user_id=42a7715a8c594a70969c2af1cf6eb188, role_id=d1109db57b004dd9a4f84a634ff76205)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:13.159450 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c552f073-7ca7-4d8f-8f2a-91576e1ad92f None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:13.171872 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 570/1123] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 10:21:13 2022] PUT /identity/v3/projects/4df4efbd6e0440b490af925e22b13188/users/42a7715a8c594a70969c2af1cf6eb188/roles/d1109db57b004dd9a4f84a634ff76205 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:21:15.245872 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685734e9-d4de-4052-a8ac-1f09e8b17595 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:15.246433 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685734e9-d4de-4052-a8ac-1f09e8b17595 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:15.246433 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685734e9-d4de-4052-a8ac-1f09e8b17595 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:15.247699 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 554/1124] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:21:15.252821 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc527b7-1ae6-4620-b0cb-3b72c6cb2d95 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:15.253238 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc527b7-1ae6-4620-b0cb-3b72c6cb2d95 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:15.253354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc527b7-1ae6-4620-b0cb-3b72c6cb2d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:15.271990 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-1dc527b7-1ae6-4620-b0cb-3b72c6cb2d95 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:21:15.273160 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-1dc527b7-1ae6-4620-b0cb-3b72c6cb2d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:21:15.336969 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 571/1125] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:21:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:21:15.598840 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685734e9-d4de-4052-a8ac-1f09e8b17595 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:15.697926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685734e9-d4de-4052-a8ac-1f09e8b17595 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:15.732602 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c44b93-4ebe-4c87-8e9b-43d28642a893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:15.733221 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c44b93-4ebe-4c87-8e9b-43d28642a893 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:15.733501 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c44b93-4ebe-4c87-8e9b-43d28642a893 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:15.733774 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c44b93-4ebe-4c87-8e9b-43d28642a893 None admin] PATH_INFO: `/v3/roles/d1109db57b004dd9a4f84a634ff76205` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:15.735188 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c44b93-4ebe-4c87-8e9b-43d28642a893 None admin] RBAC: Authorizing `identity:get_role(role_id=d1109db57b004dd9a4f84a634ff76205)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:15.736547 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c44b93-4ebe-4c87-8e9b-43d28642a893 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:15.737835 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 555/1126] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:21:15 2022] GET /identity/v3/roles/d1109db57b004dd9a4f84a634ff76205 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:15.743440 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc527b7-1ae6-4620-b0cb-3b72c6cb2d95 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:15.773302 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:15.842243 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc527b7-1ae6-4620-b0cb-3b72c6cb2d95 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:15.874572 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532720c5-7de6-4f80-a1e2-3789a5946f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:15.875139 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532720c5-7de6-4f80-a1e2-3789a5946f19 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:15.875417 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532720c5-7de6-4f80-a1e2-3789a5946f19 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:15.875691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532720c5-7de6-4f80-a1e2-3789a5946f19 None admin] PATH_INFO: `/v3/users/42a7715a8c594a70969c2af1cf6eb188` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:15.876807 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532720c5-7de6-4f80-a1e2-3789a5946f19 None admin] RBAC: Authorizing `identity:get_user(user_id=42a7715a8c594a70969c2af1cf6eb188)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:15.878261 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532720c5-7de6-4f80-a1e2-3789a5946f19 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:15.879830 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 572/1127] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:21:15 2022] GET /identity/v3/users/42a7715a8c594a70969c2af1cf6eb188 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:15.885550 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c44b93-4ebe-4c87-8e9b-43d28642a893 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:15.900413 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:15.983932 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c44b93-4ebe-4c87-8e9b-43d28642a893 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:16.017829 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd786e3-7b73-4d6b-adcf-b4ed6d4d3dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:16.018446 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd786e3-7b73-4d6b-adcf-b4ed6d4d3dfc None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:16.018728 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd786e3-7b73-4d6b-adcf-b4ed6d4d3dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:16.018998 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd786e3-7b73-4d6b-adcf-b4ed6d4d3dfc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:16.084065 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd786e3-7b73-4d6b-adcf-b4ed6d4d3dfc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:16.085492 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd786e3-7b73-4d6b-adcf-b4ed6d4d3dfc None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:16.099077 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:16.181369 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 556/1128] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:21:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 10:21:16.187678 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532720c5-7de6-4f80-a1e2-3789a5946f19 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:16.281741 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:16.285164 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532720c5-7de6-4f80-a1e2-3789a5946f19 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:16.318129 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84cecf-36bc-4706-bb72-45604dd1d7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:16.318742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84cecf-36bc-4706-bb72-45604dd1d7bf None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:16.319020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84cecf-36bc-4706-bb72-45604dd1d7bf None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:16.319291 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84cecf-36bc-4706-bb72-45604dd1d7bf None admin] PATH_INFO: `/v3/projects/4df4efbd6e0440b490af925e22b13188` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:16.320385 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84cecf-36bc-4706-bb72-45604dd1d7bf None admin] RBAC: Authorizing `identity:get_project(project_id=4df4efbd6e0440b490af925e22b13188)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:16.321794 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84cecf-36bc-4706-bb72-45604dd1d7bf None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:16.323420 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 573/1129] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:21:16 2022] GET /identity/v3/projects/4df4efbd6e0440b490af925e22b13188 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:16.330084 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd786e3-7b73-4d6b-adcf-b4ed6d4d3dfc None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:16.429734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd786e3-7b73-4d6b-adcf-b4ed6d4d3dfc None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:16.442589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:16.463886 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3872a-c167-49fd-aacf-652e3b042592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:16.464483 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3872a-c167-49fd-aacf-652e3b042592 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:16.464764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3872a-c167-49fd-aacf-652e3b042592 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:16.465037 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3872a-c167-49fd-aacf-652e3b042592 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:16.466291 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3872a-c167-49fd-aacf-652e3b042592 None admin] RBAC: Adding query filter params (role.id=d1109db57b004dd9a4f84a634ff76205, scope.project.id=4df4efbd6e0440b490af925e22b13188, user.id=42a7715a8c594a70969c2af1cf6eb188) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:21:16.466601 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3872a-c167-49fd-aacf-652e3b042592 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:16.468023 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3872a-c167-49fd-aacf-652e3b042592 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:16.473664 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 557/1130] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:21:16 2022] GET /identity/v3/role_assignments?user.id=42a7715a8c594a70969c2af1cf6eb188&scope.project.id=4df4efbd6e0440b490af925e22b13188&role.id=d1109db57b004dd9a4f84a634ff76205 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:18.624488 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e577cf-2642-4ac0-a48b-862a267d63d9 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:18.625018 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e577cf-2642-4ac0-a48b-862a267d63d9 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:18.625018 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e577cf-2642-4ac0-a48b-862a267d63d9 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:18.626412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 574/1131] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:21:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:21:18.631833 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f33b2-0be1-4783-9bc9-9b1e413dc5a3 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:18.632231 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f33b2-0be1-4783-9bc9-9b1e413dc5a3 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:18.632373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f33b2-0be1-4783-9bc9-9b1e413dc5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:18.649916 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-007f33b2-0be1-4783-9bc9-9b1e413dc5a3 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:21:18.651111 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-007f33b2-0be1-4783-9bc9-9b1e413dc5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:21:18.714781 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 558/1132] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:21:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:21:18.986215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e577cf-2642-4ac0-a48b-862a267d63d9 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:19.085301 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e577cf-2642-4ac0-a48b-862a267d63d9 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:19.118574 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309736ea-5fba-48b3-985d-1638bbfe2c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:19.119187 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309736ea-5fba-48b3-985d-1638bbfe2c3e None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:19.119461 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309736ea-5fba-48b3-985d-1638bbfe2c3e None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:19.119726 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309736ea-5fba-48b3-985d-1638bbfe2c3e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:19.120809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309736ea-5fba-48b3-985d-1638bbfe2c3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:19.122757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309736ea-5fba-48b3-985d-1638bbfe2c3e None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:19.124299 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 575/1133] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:21:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:19.129983 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f33b2-0be1-4783-9bc9-9b1e413dc5a3 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:19.227743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f33b2-0be1-4783-9bc9-9b1e413dc5a3 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:19.234726 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:19.260990 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429f45ef-9653-4562-950d-9fc515b637bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:19.261590 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f45ef-9653-4562-950d-9fc515b637bb None admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:19.261867 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f45ef-9653-4562-950d-9fc515b637bb None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:19.262169 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f45ef-9653-4562-950d-9fc515b637bb None admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:19.263428 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429f45ef-9653-4562-950d-9fc515b637bb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:19.264848 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429f45ef-9653-4562-950d-9fc515b637bb None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:19.290651 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 559/1134] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Sep 26 10:21:19 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:21:21.426905 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e202099-7248-4ee9-bfee-1c2ab2c18d4c None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:21.427457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e202099-7248-4ee9-bfee-1c2ab2c18d4c None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:21.427457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e202099-7248-4ee9-bfee-1c2ab2c18d4c None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:21.428812 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 576/1135] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:21:21.434082 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73e857-053f-4a4f-8fe6-5bb0201d4b59 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:21.435062 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73e857-053f-4a4f-8fe6-5bb0201d4b59 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:21.435163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73e857-053f-4a4f-8fe6-5bb0201d4b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:21.453352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-ee73e857-053f-4a4f-8fe6-5bb0201d4b59 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:21:21.454559 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-ee73e857-053f-4a4f-8fe6-5bb0201d4b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:21:21.518475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 560/1136] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:21:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:21:21.785799 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e202099-7248-4ee9-bfee-1c2ab2c18d4c None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:21.822035 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:21.885471 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e202099-7248-4ee9-bfee-1c2ab2c18d4c None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:21.917313 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3966f-61bf-42e3-accc-c95d137e886f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:21.917925 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3966f-61bf-42e3-accc-c95d137e886f None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:21.918234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3966f-61bf-42e3-accc-c95d137e886f None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:21.918509 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3966f-61bf-42e3-accc-c95d137e886f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:21.920131 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3966f-61bf-42e3-accc-c95d137e886f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:21.921551 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3966f-61bf-42e3-accc-c95d137e886f None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:21.923139 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 577/1137] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:21:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:21.928946 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73e857-053f-4a4f-8fe6-5bb0201d4b59 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:22.015199 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:22.027717 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73e857-053f-4a4f-8fe6-5bb0201d4b59 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:22.060277 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de1c26-641a-4c2d-b77a-ea84b44b0d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:22.060814 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de1c26-641a-4c2d-b77a-ea84b44b0d4a None admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:22.061091 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de1c26-641a-4c2d-b77a-ea84b44b0d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:22.061393 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de1c26-641a-4c2d-b77a-ea84b44b0d4a None admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:22.062055 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de1c26-641a-4c2d-b77a-ea84b44b0d4a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:22.063478 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de1c26-641a-4c2d-b77a-ea84b44b0d4a None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:22.084231 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 561/1138] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 10:21:21 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:21:24.249163 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b083d-dde5-499c-8cad-54006cb009bc None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:24.249684 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b083d-dde5-499c-8cad-54006cb009bc None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:24.249684 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b083d-dde5-499c-8cad-54006cb009bc None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:24.250988 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 578/1139] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:21:24.256312 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eaea60-5d45-429d-91bf-311ad27cb87a None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:24.256724 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eaea60-5d45-429d-91bf-311ad27cb87a None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:24.256854 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eaea60-5d45-429d-91bf-311ad27cb87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:24.274600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-44eaea60-5d45-429d-91bf-311ad27cb87a None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:21:24.275788 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-44eaea60-5d45-429d-91bf-311ad27cb87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:21:24.340282 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 562/1140] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:21:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:21:24.613984 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b083d-dde5-499c-8cad-54006cb009bc None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:24.713188 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b083d-dde5-499c-8cad-54006cb009bc None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:24.716110 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:24.747007 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5184f5-9dab-40d8-bb11-bac38c892172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:24.747615 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5184f5-9dab-40d8-bb11-bac38c892172 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:24.747890 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5184f5-9dab-40d8-bb11-bac38c892172 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:24.748158 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5184f5-9dab-40d8-bb11-bac38c892172 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:24.757830 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5184f5-9dab-40d8-bb11-bac38c892172 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:21:24.763105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-eb5184f5-9dab-40d8-bb11-bac38c892172 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:21:24.763105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:21:24.763105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:21:24.763105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:21:24.763105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:21:24.763105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:21:24.763105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:21:24.763105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:21:24.763105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:21:24.763105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:21:24.763105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:21:24.763105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:21:24.763105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:21:24.763105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:21:24.763105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:21:24.763105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:21:24.764286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:21:24.764286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:21:24.764286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:21:24.764286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:21:24.764286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:21:24.764286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:21:24.764286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:21:24.764286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:21:24.764286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:21:24.764286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:21:24.764286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:21:24.764286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:21:24.764286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:21:24.764286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:21:24.764286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:21:24.764286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:21:24.765530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:21:24.765530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:21:24.765530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:21:24.765530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:21:24.765530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:21:24.765530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:21:24.765530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:21:24.765530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:21:24.765530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:21:24.765530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:21:24.765530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:21:24.765530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:21:24.765530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 579/1141] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:21:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:21:24.770150 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eaea60-5d45-429d-91bf-311ad27cb87a None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:24.847448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:24.872364 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eaea60-5d45-429d-91bf-311ad27cb87a None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:24.905555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f4a65-1184-4b67-a393-284dbd742a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:24.906182 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f4a65-1184-4b67-a393-284dbd742a94 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:24.906460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f4a65-1184-4b67-a393-284dbd742a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:24.906730 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f4a65-1184-4b67-a393-284dbd742a94 None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:24.907380 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784f4a65-1184-4b67-a393-284dbd742a94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:21:24.907677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784f4a65-1184-4b67-a393-284dbd742a94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:24.909033 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784f4a65-1184-4b67-a393-284dbd742a94 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:24.916707 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 563/1142] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:21:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:21:24.922681 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5184f5-9dab-40d8-bb11-bac38c892172 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:25.019734 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5184f5-9dab-40d8-bb11-bac38c892172 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:25.052539 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65980804-e79d-47bd-8090-55801cddce8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:25.053132 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65980804-e79d-47bd-8090-55801cddce8d None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:25.053406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65980804-e79d-47bd-8090-55801cddce8d None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:25.053671 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65980804-e79d-47bd-8090-55801cddce8d None admin] PATH_INFO: `/v3/users/098ed8e3d3ae4654bda796cd792c626b` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:25.066194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65980804-e79d-47bd-8090-55801cddce8d None admin] RBAC: Authorizing `identity:get_user(user_id=098ed8e3d3ae4654bda796cd792c626b)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:25.067609 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65980804-e79d-47bd-8090-55801cddce8d None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:25.069162 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 580/1143] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:21:24 2022] GET /identity/v3/users/098ed8e3d3ae4654bda796cd792c626b => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:25.074773 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f4a65-1184-4b67-a393-284dbd742a94 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:25.172002 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f4a65-1184-4b67-a393-284dbd742a94 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:25.204683 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0f9d0-61ad-4226-919a-1b9af0b76105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:25.205278 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0f9d0-61ad-4226-919a-1b9af0b76105 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:25.205557 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0f9d0-61ad-4226-919a-1b9af0b76105 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:25.205830 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0f9d0-61ad-4226-919a-1b9af0b76105 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:25.209469 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:25.271346 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0f9d0-61ad-4226-919a-1b9af0b76105 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:25.272780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0f9d0-61ad-4226-919a-1b9af0b76105 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:25.366618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 564/1144] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:21:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 10:21:25.373056 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65980804-e79d-47bd-8090-55801cddce8d None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:25.397327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:25.471107 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65980804-e79d-47bd-8090-55801cddce8d None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:25.503856 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88516a-bee2-4263-ae9b-e2ae3ad8b90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:25.504472 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88516a-bee2-4263-ae9b-e2ae3ad8b90f None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:25.504753 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88516a-bee2-4263-ae9b-e2ae3ad8b90f None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:25.505027 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88516a-bee2-4263-ae9b-e2ae3ad8b90f None admin] PATH_INFO: `/v3/projects/a1f7c771fe72471cb4bb1bdccb0b98f3` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:25.506173 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c88516a-bee2-4263-ae9b-e2ae3ad8b90f None admin] RBAC: Authorizing `identity:get_project(project_id=a1f7c771fe72471cb4bb1bdccb0b98f3)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:25.507587 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c88516a-bee2-4263-ae9b-e2ae3ad8b90f None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:25.509195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 581/1145] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:21:25 2022] GET /identity/v3/projects/a1f7c771fe72471cb4bb1bdccb0b98f3 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:25.515772 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0f9d0-61ad-4226-919a-1b9af0b76105 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:25.548666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:25.611922 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0f9d0-61ad-4226-919a-1b9af0b76105 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:25.645395 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78997861-15f6-4ba8-96a5-4e39ab3da7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:25.645986 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78997861-15f6-4ba8-96a5-4e39ab3da7bd None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:25.646295 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78997861-15f6-4ba8-96a5-4e39ab3da7bd None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:25.646567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78997861-15f6-4ba8-96a5-4e39ab3da7bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:25.647283 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78997861-15f6-4ba8-96a5-4e39ab3da7bd None admin] RBAC: Adding query filter params (role.id=532b37d9076c45618cf352aafb85f35e, scope.project.id=a1f7c771fe72471cb4bb1bdccb0b98f3, user.id=098ed8e3d3ae4654bda796cd792c626b) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:21:25.647595 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78997861-15f6-4ba8-96a5-4e39ab3da7bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:25.649008 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78997861-15f6-4ba8-96a5-4e39ab3da7bd None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:25.654229 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 565/1146] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:21:25 2022] GET /identity/v3/role_assignments?user.id=098ed8e3d3ae4654bda796cd792c626b&scope.project.id=a1f7c771fe72471cb4bb1bdccb0b98f3&role.id=532b37d9076c45618cf352aafb85f35e => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:27.747950 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c343a3-5c11-4881-b9bc-e30759151bcc None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:27.748504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c343a3-5c11-4881-b9bc-e30759151bcc None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:27.748504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c343a3-5c11-4881-b9bc-e30759151bcc None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:27.749830 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 582/1147] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:21:27.754885 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23146bbe-b70c-4c52-9f69-222a2ed10693 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:27.755329 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23146bbe-b70c-4c52-9f69-222a2ed10693 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:27.755424 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23146bbe-b70c-4c52-9f69-222a2ed10693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:27.773665 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-23146bbe-b70c-4c52-9f69-222a2ed10693 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:21:27.774890 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-23146bbe-b70c-4c52-9f69-222a2ed10693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:21:27.841421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 566/1148] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:21:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:21:28.108112 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c343a3-5c11-4881-b9bc-e30759151bcc None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:28.196046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:28.207139 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c343a3-5c11-4881-b9bc-e30759151bcc None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:28.239500 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e20a7-aad4-45ea-917f-67d54b97c5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:28.240099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e20a7-aad4-45ea-917f-67d54b97c5da None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:28.240373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e20a7-aad4-45ea-917f-67d54b97c5da None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:28.240638 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e20a7-aad4-45ea-917f-67d54b97c5da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:28.251078 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01e20a7-aad4-45ea-917f-67d54b97c5da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:21:28.255819 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-c01e20a7-aad4-45ea-917f-67d54b97c5da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:21:28.255819 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:21:28.255819 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:21:28.255819 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:21:28.255819 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:21:28.255819 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:21:28.255819 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:21:28.255819 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:21:28.255819 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:21:28.255819 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:21:28.255819 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:21:28.255819 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:21:28.255819 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:21:28.255819 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:21:28.255819 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:21:28.255819 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:21:28.256939 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:21:28.256939 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:21:28.256939 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:21:28.256939 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:21:28.256939 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:21:28.256939 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:21:28.256939 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:21:28.256939 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:21:28.256939 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:21:28.256939 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:21:28.256939 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:21:28.256939 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:21:28.256939 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:21:28.256939 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:21:28.256939 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:21:28.256939 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:21:28.258014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:21:28.258014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:21:28.258014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:21:28.258014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:21:28.258014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:21:28.258014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:21:28.258014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:21:28.258014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:21:28.258014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:21:28.258014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:21:28.258014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:21:28.258014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:21:28.258014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 583/1149] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:21:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:21:28.263552 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23146bbe-b70c-4c52-9f69-222a2ed10693 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:28.292255 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:28.358755 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23146bbe-b70c-4c52-9f69-222a2ed10693 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:28.395740 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4af052-5e9b-4789-aecd-9fffdc81b96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:28.396332 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4af052-5e9b-4789-aecd-9fffdc81b96f None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:28.396605 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4af052-5e9b-4789-aecd-9fffdc81b96f None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:28.396870 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4af052-5e9b-4789-aecd-9fffdc81b96f None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:28.397514 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4af052-5e9b-4789-aecd-9fffdc81b96f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:21:28.397811 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4af052-5e9b-4789-aecd-9fffdc81b96f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:28.399196 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4af052-5e9b-4789-aecd-9fffdc81b96f None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:28.407002 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 567/1150] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:21:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:28.412739 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e20a7-aad4-45ea-917f-67d54b97c5da None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:28.509656 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e20a7-aad4-45ea-917f-67d54b97c5da None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:28.542316 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacae06b-50f1-4364-96c3-2e868c5a9b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:28.542922 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacae06b-50f1-4364-96c3-2e868c5a9b12 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:28.543199 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacae06b-50f1-4364-96c3-2e868c5a9b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:28.543471 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacae06b-50f1-4364-96c3-2e868c5a9b12 None admin] PATH_INFO: `/v3/users/098ed8e3d3ae4654bda796cd792c626b` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:28.545083 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacae06b-50f1-4364-96c3-2e868c5a9b12 None admin] RBAC: Authorizing `identity:get_user(user_id=098ed8e3d3ae4654bda796cd792c626b)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:28.546532 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacae06b-50f1-4364-96c3-2e868c5a9b12 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:28.548096 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 584/1151] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:21:28 2022] GET /identity/v3/users/098ed8e3d3ae4654bda796cd792c626b => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:28.553704 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4af052-5e9b-4789-aecd-9fffdc81b96f None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:28.642067 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:28.651174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4af052-5e9b-4789-aecd-9fffdc81b96f None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:28.683728 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2fa79-13e0-4359-a24d-fdf4e395ad3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:28.684928 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2fa79-13e0-4359-a24d-fdf4e395ad3f None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:28.685207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2fa79-13e0-4359-a24d-fdf4e395ad3f None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:28.685476 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2fa79-13e0-4359-a24d-fdf4e395ad3f None admin] PATH_INFO: `/v3/projects/a1f7c771fe72471cb4bb1bdccb0b98f3` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:28.686599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a2fa79-13e0-4359-a24d-fdf4e395ad3f None admin] RBAC: Authorizing `identity:get_project(project_id=a1f7c771fe72471cb4bb1bdccb0b98f3)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:28.688012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a2fa79-13e0-4359-a24d-fdf4e395ad3f None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:28.689602 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 568/1152] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:21:28 2022] GET /identity/v3/projects/a1f7c771fe72471cb4bb1bdccb0b98f3 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:28.695305 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacae06b-50f1-4364-96c3-2e868c5a9b12 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:28.714199 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:28.791839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacae06b-50f1-4364-96c3-2e868c5a9b12 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:28.825718 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6bc987-4b92-4025-a5ea-9c05520fb1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:28.826404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bc987-4b92-4025-a5ea-9c05520fb1b3 None admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:28.826688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bc987-4b92-4025-a5ea-9c05520fb1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:28.826962 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bc987-4b92-4025-a5ea-9c05520fb1b3 None admin] PATH_INFO: `/v3/projects/a1f7c771fe72471cb4bb1bdccb0b98f3/users/098ed8e3d3ae4654bda796cd792c626b/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:28.828855 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6bc987-4b92-4025-a5ea-9c05520fb1b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a1f7c771fe72471cb4bb1bdccb0b98f3, user_id=098ed8e3d3ae4654bda796cd792c626b, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:28.830306 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6bc987-4b92-4025-a5ea-9c05520fb1b3 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:28.843306 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 585/1153] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 10:21:28 2022] PUT /identity/v3/projects/a1f7c771fe72471cb4bb1bdccb0b98f3/users/098ed8e3d3ae4654bda796cd792c626b/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:21:30.948919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b10c2d-8b3b-4da6-9d91-73aa3ee2a6b3 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:30.949437 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b10c2d-8b3b-4da6-9d91-73aa3ee2a6b3 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:30.949437 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b10c2d-8b3b-4da6-9d91-73aa3ee2a6b3 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:30.950829 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 569/1154] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:21:30.956035 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed148ad-4409-46fd-bddc-1e3b2d3918d7 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:30.956418 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed148ad-4409-46fd-bddc-1e3b2d3918d7 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:30.956585 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed148ad-4409-46fd-bddc-1e3b2d3918d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:30.975233 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-bed148ad-4409-46fd-bddc-1e3b2d3918d7 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:21:30.976414 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-bed148ad-4409-46fd-bddc-1e3b2d3918d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:21:31.040382 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 586/1155] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:21:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:21:31.309477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b10c2d-8b3b-4da6-9d91-73aa3ee2a6b3 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:31.408425 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b10c2d-8b3b-4da6-9d91-73aa3ee2a6b3 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:31.442127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc456bb5-66b4-47df-9ab7-79db524ea5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:31.442742 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc456bb5-66b4-47df-9ab7-79db524ea5b0 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:31.443017 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc456bb5-66b4-47df-9ab7-79db524ea5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:31.443285 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc456bb5-66b4-47df-9ab7-79db524ea5b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:31.453438 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc456bb5-66b4-47df-9ab7-79db524ea5b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:21:31.458720 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-cc456bb5-66b4-47df-9ab7-79db524ea5b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:21:31.458720 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:21:31.458720 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:21:31.458720 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:21:31.458720 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:21:31.458720 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:21:31.458720 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:21:31.458720 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:21:31.458720 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:21:31.458720 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:21:31.458720 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:21:31.458720 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:21:31.458720 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:21:31.458720 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:21:31.458720 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:21:31.458720 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:21:31.459796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:21:31.459796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:21:31.459796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:21:31.459796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:21:31.459796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:21:31.459796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:21:31.459796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:21:31.459796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:21:31.459796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:21:31.459796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:21:31.459796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:21:31.459796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:21:31.459796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:21:31.459796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:21:31.459796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:21:31.459796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:21:31.460858 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:21:31.460858 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:21:31.460858 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:21:31.460858 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:21:31.460858 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:21:31.460858 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:21:31.460858 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:21:31.460858 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:21:31.460858 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:21:31.460858 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:21:31.460858 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:21:31.460858 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:21:31.460858 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 570/1156] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:21:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:21:31.466489 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed148ad-4409-46fd-bddc-1e3b2d3918d7 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:31.476429 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:31.562970 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed148ad-4409-46fd-bddc-1e3b2d3918d7 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:31.596259 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d2a25-bbe0-4d53-8b15-4e68fbc82a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:31.596852 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d2a25-bbe0-4d53-8b15-4e68fbc82a35 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:31.597124 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d2a25-bbe0-4d53-8b15-4e68fbc82a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:31.597392 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d2a25-bbe0-4d53-8b15-4e68fbc82a35 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:31.598059 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2d2a25-bbe0-4d53-8b15-4e68fbc82a35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:21:31.598363 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2d2a25-bbe0-4d53-8b15-4e68fbc82a35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:31.599703 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2d2a25-bbe0-4d53-8b15-4e68fbc82a35 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:31.607352 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 587/1157] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:21:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:31.613141 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc456bb5-66b4-47df-9ab7-79db524ea5b0 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:31.634334 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:31.709203 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc456bb5-66b4-47df-9ab7-79db524ea5b0 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:31.740934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7aaac-9503-4478-9a89-4d6367ecd4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:31.741537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7aaac-9503-4478-9a89-4d6367ecd4e7 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:31.741814 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7aaac-9503-4478-9a89-4d6367ecd4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:31.742106 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7aaac-9503-4478-9a89-4d6367ecd4e7 None admin] PATH_INFO: `/v3/users/098ed8e3d3ae4654bda796cd792c626b` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:31.743190 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d7aaac-9503-4478-9a89-4d6367ecd4e7 None admin] RBAC: Authorizing `identity:get_user(user_id=098ed8e3d3ae4654bda796cd792c626b)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:31.744590 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d7aaac-9503-4478-9a89-4d6367ecd4e7 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:31.746139 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 571/1158] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:21:31 2022] GET /identity/v3/users/098ed8e3d3ae4654bda796cd792c626b => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:31.751892 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d2a25-bbe0-4d53-8b15-4e68fbc82a35 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:31.819188 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:31.850839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d2a25-bbe0-4d53-8b15-4e68fbc82a35 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:31.883569 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8cc61d-d900-48f5-a9ce-183e67098a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:31.884174 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8cc61d-d900-48f5-a9ce-183e67098a7d None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:31.884453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8cc61d-d900-48f5-a9ce-183e67098a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:31.884723 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8cc61d-d900-48f5-a9ce-183e67098a7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:31.954909 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8cc61d-d900-48f5-a9ce-183e67098a7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:31.956341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8cc61d-d900-48f5-a9ce-183e67098a7d None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:32.021689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:32.052126 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 588/1159] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:21:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 10:21:32.058170 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7aaac-9503-4478-9a89-4d6367ecd4e7 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:32.143858 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:32.155446 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7aaac-9503-4478-9a89-4d6367ecd4e7 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:32.187080 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bec1a3-4ab6-4c33-ad54-de5d06db8600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:32.187689 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bec1a3-4ab6-4c33-ad54-de5d06db8600 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:32.187959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bec1a3-4ab6-4c33-ad54-de5d06db8600 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:32.188221 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bec1a3-4ab6-4c33-ad54-de5d06db8600 None admin] PATH_INFO: `/v3/projects/a1f7c771fe72471cb4bb1bdccb0b98f3` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:32.189305 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bec1a3-4ab6-4c33-ad54-de5d06db8600 None admin] RBAC: Authorizing `identity:get_project(project_id=a1f7c771fe72471cb4bb1bdccb0b98f3)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:32.190772 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bec1a3-4ab6-4c33-ad54-de5d06db8600 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:32.192340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 572/1160] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:21:32 2022] GET /identity/v3/projects/a1f7c771fe72471cb4bb1bdccb0b98f3 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:32.198424 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8cc61d-d900-48f5-a9ce-183e67098a7d None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:32.303194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8cc61d-d900-48f5-a9ce-183e67098a7d None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:32.336904 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ca09f-3aac-4182-be95-337e9e7288b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:32.337500 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ca09f-3aac-4182-be95-337e9e7288b5 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:32.337775 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ca09f-3aac-4182-be95-337e9e7288b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:32.338067 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ca09f-3aac-4182-be95-337e9e7288b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:32.338782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822ca09f-3aac-4182-be95-337e9e7288b5 None admin] RBAC: Adding query filter params (role.id=532b37d9076c45618cf352aafb85f35e, scope.project.id=a1f7c771fe72471cb4bb1bdccb0b98f3, user.id=098ed8e3d3ae4654bda796cd792c626b) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:21:32.339083 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822ca09f-3aac-4182-be95-337e9e7288b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:32.340509 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822ca09f-3aac-4182-be95-337e9e7288b5 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:32.346091 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 589/1161] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:21:32 2022] GET /identity/v3/role_assignments?user.id=098ed8e3d3ae4654bda796cd792c626b&scope.project.id=a1f7c771fe72471cb4bb1bdccb0b98f3&role.id=532b37d9076c45618cf352aafb85f35e => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:34.512019 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f62cd-824c-40da-8bc1-abbbedf20834 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:34.513187 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f62cd-824c-40da-8bc1-abbbedf20834 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:34.513187 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f62cd-824c-40da-8bc1-abbbedf20834 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:34.514465 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 573/1162] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:21:34.519702 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877f331-e271-454e-a17d-177929e91e58 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:34.520114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877f331-e271-454e-a17d-177929e91e58 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:34.520251 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877f331-e271-454e-a17d-177929e91e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:34.537906 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-6877f331-e271-454e-a17d-177929e91e58 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:21:34.539117 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-6877f331-e271-454e-a17d-177929e91e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:21:34.565677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:34.605331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 590/1163] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:21:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:21:34.873647 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f62cd-824c-40da-8bc1-abbbedf20834 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:34.980098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f62cd-824c-40da-8bc1-abbbedf20834 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:35.012972 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26687e7-b963-4dc2-bf39-e57a4cc006a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:35.013568 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26687e7-b963-4dc2-bf39-e57a4cc006a8 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:35.013844 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26687e7-b963-4dc2-bf39-e57a4cc006a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:35.014141 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26687e7-b963-4dc2-bf39-e57a4cc006a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:35.037596 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:35.080005 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26687e7-b963-4dc2-bf39-e57a4cc006a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:35.081370 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26687e7-b963-4dc2-bf39-e57a4cc006a8 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:35.175239 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 574/1164] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:21:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 10:21:35.181162 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877f331-e271-454e-a17d-177929e91e58 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:35.291368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877f331-e271-454e-a17d-177929e91e58 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:35.326077 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f181ad1-b39d-44cf-9a4f-b4c0bcf305b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:35.326697 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f181ad1-b39d-44cf-9a4f-b4c0bcf305b6 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:35.326974 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f181ad1-b39d-44cf-9a4f-b4c0bcf305b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:35.327250 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f181ad1-b39d-44cf-9a4f-b4c0bcf305b6 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:35.334004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f181ad1-b39d-44cf-9a4f-b4c0bcf305b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:35.335405 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f181ad1-b39d-44cf-9a4f-b4c0bcf305b6 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:35.346518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-1f181ad1-b39d-44cf-9a4f-b4c0bcf305b6 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 26 10:21:35.346518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:21:35.346518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 26 10:21:35.346518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 26 10:21:35.346518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:21:35.346518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:21:35.346518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:21:35.346518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:21:35.346518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 26 10:21:35.346518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:21:35.346518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 10:21:35.346518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:21:35.346518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:21:35.346518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:21:35.346518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:21:35.346518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:21:35.346518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:21:35.347765 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:21:35.347765 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:21:35.347765 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:21:35.347765 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:21:35.347765 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:21:35.347765 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:21:35.347765 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:21:35.347765 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:21:35.347765 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:21:35.347765 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:21:35.347765 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:21:35.347765 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:21:35.347765 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:21:35.347765 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:21:35.347765 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:21:35.347765 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:21:35.348839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:21:35.348839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:21:35.348839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:21:35.348839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:21:35.348839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:21:35.348839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:21:35.348839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:21:35.348839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:21:35.348839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:21:35.348839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:21:35.348839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 26 10:21:35.348839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:21:35.348839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 26 10:21:35.348839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:21:35.348839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 591/1165] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Sep 26 10:21:35 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:21:35.353565 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26687e7-b963-4dc2-bf39-e57a4cc006a8 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:35.408494 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:35.451014 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26687e7-b963-4dc2-bf39-e57a4cc006a8 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:35.484052 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114d789-3864-4028-b7a4-b2e0a0fd79bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:35.484645 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114d789-3864-4028-b7a4-b2e0a0fd79bd None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:35.484919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114d789-3864-4028-b7a4-b2e0a0fd79bd None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:35.485188 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114d789-3864-4028-b7a4-b2e0a0fd79bd None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:35.485815 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114d789-3864-4028-b7a4-b2e0a0fd79bd None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:21:35.486139 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114d789-3864-4028-b7a4-b2e0a0fd79bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:35.487482 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114d789-3864-4028-b7a4-b2e0a0fd79bd None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:35.494319 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 575/1166] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Sep 26 10:21:35 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:37.603928 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa317122-7381-4406-9627-188e74872390 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:37.604486 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa317122-7381-4406-9627-188e74872390 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:37.604486 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa317122-7381-4406-9627-188e74872390 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:37.605788 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 592/1167] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:21:37.610987 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c8a31-460a-4ad7-b852-d004faba0cda None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:37.611384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c8a31-460a-4ad7-b852-d004faba0cda None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:37.611532 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c8a31-460a-4ad7-b852-d004faba0cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:37.630192 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-2a2c8a31-460a-4ad7-b852-d004faba0cda None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:21:37.631535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-2a2c8a31-460a-4ad7-b852-d004faba0cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:21:37.695728 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 576/1168] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:21:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:21:37.972825 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa317122-7381-4406-9627-188e74872390 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:37.997917 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:38.070689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa317122-7381-4406-9627-188e74872390 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:38.102937 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9d115-68cc-48cb-902b-d68567c6ed8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:38.103520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9d115-68cc-48cb-902b-d68567c6ed8d None admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:38.103795 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9d115-68cc-48cb-902b-d68567c6ed8d None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:38.104061 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9d115-68cc-48cb-902b-d68567c6ed8d None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:38.104715 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd9d115-68cc-48cb-902b-d68567c6ed8d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:38.106131 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd9d115-68cc-48cb-902b-d68567c6ed8d None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:38.120559 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 593/1169] 10.222.0.49 () {60 vars in 1138 bytes} [Mon Sep 26 10:21:37 2022] POST /identity/v3/domains => generated 267 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:21:40.315056 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb5d4d-2aa3-40e3-8ab4-6331853ad59a None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:40.315611 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb5d4d-2aa3-40e3-8ab4-6331853ad59a None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:40.315611 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb5d4d-2aa3-40e3-8ab4-6331853ad59a None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:40.316923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 577/1170] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:21:40.322368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae4d04-ae9b-471f-9973-578706a34a3b None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:40.322813 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae4d04-ae9b-471f-9973-578706a34a3b None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:40.322919 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae4d04-ae9b-471f-9973-578706a34a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:40.345047 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-5fae4d04-ae9b-471f-9973-578706a34a3b None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:21:40.346374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-5fae4d04-ae9b-471f-9973-578706a34a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:21:40.421875 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 594/1171] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:21:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:21:40.690479 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb5d4d-2aa3-40e3-8ab4-6331853ad59a None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:40.739879 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:40.788831 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb5d4d-2aa3-40e3-8ab4-6331853ad59a None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:40.822488 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f104b88-ddcb-4539-afc0-eee61270411b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:40.823115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f104b88-ddcb-4539-afc0-eee61270411b None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:40.823392 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f104b88-ddcb-4539-afc0-eee61270411b None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:40.823660 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f104b88-ddcb-4539-afc0-eee61270411b None admin] PATH_INFO: `/v3/domains/c4c9d5113f7f4fbb8423245a428e36d3` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:40.834548 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f104b88-ddcb-4539-afc0-eee61270411b None admin] RBAC: Authorizing `identity:get_domain(domain_id=c4c9d5113f7f4fbb8423245a428e36d3)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:40.835984 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f104b88-ddcb-4539-afc0-eee61270411b None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:40.837563 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 578/1172] 10.222.0.49 () {56 vars in 1183 bytes} [Mon Sep 26 10:21:40 2022] GET /identity/v3/domains/c4c9d5113f7f4fbb8423245a428e36d3 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:40.844034 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae4d04-ae9b-471f-9973-578706a34a3b None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:40.858642 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:40.941346 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae4d04-ae9b-471f-9973-578706a34a3b None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:40.974415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1609143-7629-4e84-8f99-fe82ac10a266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:40.975011 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1609143-7629-4e84-8f99-fe82ac10a266 None admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:40.975288 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1609143-7629-4e84-8f99-fe82ac10a266 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:40.975556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1609143-7629-4e84-8f99-fe82ac10a266 None admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:40.976200 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1609143-7629-4e84-8f99-fe82ac10a266 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:40.977608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1609143-7629-4e84-8f99-fe82ac10a266 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:41.003056 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 595/1173] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Sep 26 10:21:40 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:21:43.140893 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21a652-ec76-486e-879a-0ca401db4cca None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:43.141409 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21a652-ec76-486e-879a-0ca401db4cca None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:43.141409 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21a652-ec76-486e-879a-0ca401db4cca None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:43.142722 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 579/1174] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:21:43.147783 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61beefd-7893-4f22-a7a0-b2c7160cbd01 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:43.148223 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61beefd-7893-4f22-a7a0-b2c7160cbd01 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:43.148331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61beefd-7893-4f22-a7a0-b2c7160cbd01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:43.166506 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-d61beefd-7893-4f22-a7a0-b2c7160cbd01 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:21:43.167668 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-d61beefd-7893-4f22-a7a0-b2c7160cbd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:21:43.227408 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:43.236500 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 596/1175] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:21:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:21:43.501362 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21a652-ec76-486e-879a-0ca401db4cca None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:43.601460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21a652-ec76-486e-879a-0ca401db4cca None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:43.611369 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:43.634250 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9ac56-3c19-4f82-b55a-99a5e096bb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:43.634874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9ac56-3c19-4f82-b55a-99a5e096bb49 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:43.635153 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9ac56-3c19-4f82-b55a-99a5e096bb49 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:43.635424 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9ac56-3c19-4f82-b55a-99a5e096bb49 None admin] PATH_INFO: `/v3/domains/c4c9d5113f7f4fbb8423245a428e36d3` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:43.636525 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9ac56-3c19-4f82-b55a-99a5e096bb49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=c4c9d5113f7f4fbb8423245a428e36d3)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:43.638499 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9ac56-3c19-4f82-b55a-99a5e096bb49 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:43.640048 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 580/1176] 10.222.0.49 () {56 vars in 1183 bytes} [Mon Sep 26 10:21:43 2022] GET /identity/v3/domains/c4c9d5113f7f4fbb8423245a428e36d3 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:43.645676 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61beefd-7893-4f22-a7a0-b2c7160cbd01 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:43.740874 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61beefd-7893-4f22-a7a0-b2c7160cbd01 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:43.773383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4694ebe1-e370-4c99-bc73-1d94a078dd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:43.773976 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4694ebe1-e370-4c99-bc73-1d94a078dd73 None admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:43.774280 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4694ebe1-e370-4c99-bc73-1d94a078dd73 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:43.774553 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4694ebe1-e370-4c99-bc73-1d94a078dd73 None admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:43.775200 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4694ebe1-e370-4c99-bc73-1d94a078dd73 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:43.776601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4694ebe1-e370-4c99-bc73-1d94a078dd73 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:43.796939 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 597/1177] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 10:21:43 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:21:45.942134 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0399b5a-7922-409f-8dfa-c17dc226119f None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:45.942645 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0399b5a-7922-409f-8dfa-c17dc226119f None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:45.942754 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0399b5a-7922-409f-8dfa-c17dc226119f None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:45.943996 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 581/1178] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:21:45.949381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceac8af-2bfb-4a6e-8ae8-b9f60538573c None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:45.950321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceac8af-2bfb-4a6e-8ae8-b9f60538573c None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:45.950490 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceac8af-2bfb-4a6e-8ae8-b9f60538573c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:45.969012 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-6ceac8af-2bfb-4a6e-8ae8-b9f60538573c None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:21:45.970229 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-6ceac8af-2bfb-4a6e-8ae8-b9f60538573c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:21:46.015308 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:46.035669 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 598/1179] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:21:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:21:46.301002 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0399b5a-7922-409f-8dfa-c17dc226119f None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:46.398992 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0399b5a-7922-409f-8dfa-c17dc226119f None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:46.431563 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0693c09-60a6-4daa-9c38-0f5e169d49eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:46.432201 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0693c09-60a6-4daa-9c38-0f5e169d49eb None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:46.432476 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0693c09-60a6-4daa-9c38-0f5e169d49eb None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:46.432742 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0693c09-60a6-4daa-9c38-0f5e169d49eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:46.443467 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0693c09-60a6-4daa-9c38-0f5e169d49eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:21:46.448175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-e0693c09-60a6-4daa-9c38-0f5e169d49eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:21:46.448175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:21:46.448175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:21:46.448175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:21:46.448175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:21:46.448175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:21:46.448175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:21:46.448175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:21:46.448175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:21:46.448175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:21:46.448175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:21:46.448175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:21:46.448175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:21:46.448175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:21:46.448175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:21:46.448175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:21:46.449286 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:21:46.449286 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:21:46.449286 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:21:46.449286 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:21:46.449286 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:21:46.449286 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:21:46.449286 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:21:46.449286 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:21:46.449286 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:21:46.449286 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:21:46.449286 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:21:46.449286 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:21:46.449286 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:21:46.449286 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:21:46.449286 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:21:46.449286 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:21:46.451092 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:21:46.451092 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:21:46.451092 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:21:46.451092 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:21:46.451092 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:21:46.451092 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:21:46.451092 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:21:46.451092 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:21:46.451092 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:21:46.451092 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:21:46.451092 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:21:46.451092 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:21:46.451092 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 582/1180] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:21:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:21:46.455745 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceac8af-2bfb-4a6e-8ae8-b9f60538573c None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:46.554756 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceac8af-2bfb-4a6e-8ae8-b9f60538573c None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:46.588148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f835c-644a-4fca-b6eb-15404529599b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:46.588738 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f835c-644a-4fca-b6eb-15404529599b None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:46.589011 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f835c-644a-4fca-b6eb-15404529599b None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:46.589280 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f835c-644a-4fca-b6eb-15404529599b None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:46.589931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f835c-644a-4fca-b6eb-15404529599b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:21:46.590255 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f835c-644a-4fca-b6eb-15404529599b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:46.591622 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f835c-644a-4fca-b6eb-15404529599b None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:46.599297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 599/1181] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:21:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:46.605233 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0693c09-60a6-4daa-9c38-0f5e169d49eb None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:46.644533 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:46.702448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0693c09-60a6-4daa-9c38-0f5e169d49eb None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:46.734918 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a8488-277c-41b2-9936-5abc3b4dafba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:46.735531 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a8488-277c-41b2-9936-5abc3b4dafba None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:46.735807 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a8488-277c-41b2-9936-5abc3b4dafba None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:46.736078 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a8488-277c-41b2-9936-5abc3b4dafba None admin] PATH_INFO: `/v3/users/a3936b76bc4e481e996fb09dc7089c50` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:46.749218 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a8488-277c-41b2-9936-5abc3b4dafba None admin] RBAC: Authorizing `identity:get_user(user_id=a3936b76bc4e481e996fb09dc7089c50)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:46.750698 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a8488-277c-41b2-9936-5abc3b4dafba None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:46.752253 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 583/1182] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:21:46 2022] GET /identity/v3/users/a3936b76bc4e481e996fb09dc7089c50 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:46.757869 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f835c-644a-4fca-b6eb-15404529599b None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:46.802583 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:46.855258 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f835c-644a-4fca-b6eb-15404529599b None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:46.887527 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d47450-c5a4-44a9-8d64-33e3d68f5564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:46.888118 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d47450-c5a4-44a9-8d64-33e3d68f5564 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:46.888392 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d47450-c5a4-44a9-8d64-33e3d68f5564 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:46.888659 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d47450-c5a4-44a9-8d64-33e3d68f5564 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:46.952104 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d47450-c5a4-44a9-8d64-33e3d68f5564 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:46.953513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d47450-c5a4-44a9-8d64-33e3d68f5564 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:46.996220 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:47.048460 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 600/1183] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:21:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 10:21:47.054319 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a8488-277c-41b2-9936-5abc3b4dafba None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:47.143882 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:47.149700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a8488-277c-41b2-9936-5abc3b4dafba None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:47.183328 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f515393-ce4a-4e86-bcae-9fe55bda368d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:47.183945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f515393-ce4a-4e86-bcae-9fe55bda368d None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:47.184225 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f515393-ce4a-4e86-bcae-9fe55bda368d None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:47.184497 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f515393-ce4a-4e86-bcae-9fe55bda368d None admin] PATH_INFO: `/v3/projects/d0a0755303374019b40301d4352eb9f1` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:47.185608 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f515393-ce4a-4e86-bcae-9fe55bda368d None admin] RBAC: Authorizing `identity:get_project(project_id=d0a0755303374019b40301d4352eb9f1)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:47.187057 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f515393-ce4a-4e86-bcae-9fe55bda368d None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:47.188659 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 584/1184] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:21:47 2022] GET /identity/v3/projects/d0a0755303374019b40301d4352eb9f1 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:47.194657 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d47450-c5a4-44a9-8d64-33e3d68f5564 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:47.292989 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d47450-c5a4-44a9-8d64-33e3d68f5564 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:47.326364 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02aad0a-41ed-4ab2-ae04-f07bffeb7566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:47.326956 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02aad0a-41ed-4ab2-ae04-f07bffeb7566 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:47.327231 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02aad0a-41ed-4ab2-ae04-f07bffeb7566 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:47.327503 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02aad0a-41ed-4ab2-ae04-f07bffeb7566 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:47.328215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02aad0a-41ed-4ab2-ae04-f07bffeb7566 None admin] RBAC: Adding query filter params (role.id=532b37d9076c45618cf352aafb85f35e, scope.project.id=d0a0755303374019b40301d4352eb9f1, user.id=a3936b76bc4e481e996fb09dc7089c50) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:21:47.328562 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02aad0a-41ed-4ab2-ae04-f07bffeb7566 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:47.329981 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02aad0a-41ed-4ab2-ae04-f07bffeb7566 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:47.337231 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 601/1185] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:21:47 2022] GET /identity/v3/role_assignments?user.id=a3936b76bc4e481e996fb09dc7089c50&scope.project.id=d0a0755303374019b40301d4352eb9f1&role.id=532b37d9076c45618cf352aafb85f35e => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:49.425543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236f99f-70e5-4260-916d-4e876a6508f0 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:49.426118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236f99f-70e5-4260-916d-4e876a6508f0 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:49.426118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236f99f-70e5-4260-916d-4e876a6508f0 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:49.427423 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 585/1186] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:21:49.432564 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766b302-1d04-469d-a49d-0f5860f7bc40 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:49.432986 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766b302-1d04-469d-a49d-0f5860f7bc40 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:49.433086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766b302-1d04-469d-a49d-0f5860f7bc40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:49.450652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-8766b302-1d04-469d-a49d-0f5860f7bc40 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:21:49.451823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-8766b302-1d04-469d-a49d-0f5860f7bc40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:21:49.453711 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:49.516793 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 602/1187] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:21:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 10:21:49.784882 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236f99f-70e5-4260-916d-4e876a6508f0 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:49.883309 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236f99f-70e5-4260-916d-4e876a6508f0 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:49.916545 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8acfa04-9eee-42b3-84fc-6ebca6c053c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:49.917146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acfa04-9eee-42b3-84fc-6ebca6c053c9 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:49.917419 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acfa04-9eee-42b3-84fc-6ebca6c053c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:49.917682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acfa04-9eee-42b3-84fc-6ebca6c053c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:49.927314 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8acfa04-9eee-42b3-84fc-6ebca6c053c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:21:49.931991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-b8acfa04-9eee-42b3-84fc-6ebca6c053c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:21:49.931991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:21:49.931991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:21:49.931991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:21:49.931991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:21:49.931991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:21:49.931991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:21:49.931991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:21:49.931991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:21:49.931991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:21:49.931991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:21:49.931991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:21:49.931991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:21:49.931991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:21:49.931991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:21:49.931991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:21:49.934639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:21:49.934639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:21:49.934639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:21:49.934639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:21:49.934639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:21:49.934639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:21:49.934639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:21:49.934639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:21:49.934639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:21:49.934639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:21:49.934639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:21:49.934639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:21:49.934639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:21:49.934639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:21:49.934639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:21:49.934639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:21:49.935747 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:21:49.935747 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:21:49.935747 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:21:49.935747 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:21:49.935747 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:21:49.935747 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:21:49.935747 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:21:49.935747 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:21:49.935747 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:21:49.935747 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:21:49.935747 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:21:49.935747 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:21:49.935747 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 586/1188] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:21:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:21:49.939601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766b302-1d04-469d-a49d-0f5860f7bc40 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:50.035126 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766b302-1d04-469d-a49d-0f5860f7bc40 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:50.067446 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48054e5-5405-453e-b03b-9de017af8a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:50.068029 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48054e5-5405-453e-b03b-9de017af8a43 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:50.068302 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48054e5-5405-453e-b03b-9de017af8a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:50.068570 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48054e5-5405-453e-b03b-9de017af8a43 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:50.069212 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48054e5-5405-453e-b03b-9de017af8a43 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:21:50.069505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48054e5-5405-453e-b03b-9de017af8a43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:50.070870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48054e5-5405-453e-b03b-9de017af8a43 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:50.076227 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:50.079730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 603/1189] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:21:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:50.085435 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8acfa04-9eee-42b3-84fc-6ebca6c053c9 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:50.109742 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:50.181416 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8acfa04-9eee-42b3-84fc-6ebca6c053c9 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:50.215179 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d09f9-d7f9-4ccf-adbf-c473e36d00f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:50.215850 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d09f9-d7f9-4ccf-adbf-c473e36d00f1 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:50.216128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d09f9-d7f9-4ccf-adbf-c473e36d00f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:50.216399 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d09f9-d7f9-4ccf-adbf-c473e36d00f1 None admin] PATH_INFO: `/v3/users/a3936b76bc4e481e996fb09dc7089c50` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:50.217509 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d09f9-d7f9-4ccf-adbf-c473e36d00f1 None admin] RBAC: Authorizing `identity:get_user(user_id=a3936b76bc4e481e996fb09dc7089c50)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:50.218947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d09f9-d7f9-4ccf-adbf-c473e36d00f1 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:50.220490 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 587/1190] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:21:50 2022] GET /identity/v3/users/a3936b76bc4e481e996fb09dc7089c50 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:50.226060 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48054e5-5405-453e-b03b-9de017af8a43 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:50.322913 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48054e5-5405-453e-b03b-9de017af8a43 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:50.360917 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3431096-eb20-4fd3-a9a1-fa6061569b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:50.361537 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3431096-eb20-4fd3-a9a1-fa6061569b95 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:50.361818 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3431096-eb20-4fd3-a9a1-fa6061569b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:50.362117 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3431096-eb20-4fd3-a9a1-fa6061569b95 None admin] PATH_INFO: `/v3/projects/d0a0755303374019b40301d4352eb9f1` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:50.363222 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3431096-eb20-4fd3-a9a1-fa6061569b95 None admin] RBAC: Authorizing `identity:get_project(project_id=d0a0755303374019b40301d4352eb9f1)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:50.364643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3431096-eb20-4fd3-a9a1-fa6061569b95 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:50.366298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 604/1191] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:21:50 2022] GET /identity/v3/projects/d0a0755303374019b40301d4352eb9f1 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:50.372875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d09f9-d7f9-4ccf-adbf-c473e36d00f1 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:50.460215 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:50.469101 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d09f9-d7f9-4ccf-adbf-c473e36d00f1 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:50.505562 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d8003e-6bfb-4260-b924-3e7c51fe5fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:50.506239 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8003e-6bfb-4260-b924-3e7c51fe5fdb None admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:50.506536 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8003e-6bfb-4260-b924-3e7c51fe5fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:50.506790 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8003e-6bfb-4260-b924-3e7c51fe5fdb None admin] PATH_INFO: `/v3/projects/d0a0755303374019b40301d4352eb9f1/users/a3936b76bc4e481e996fb09dc7089c50/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:50.508674 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d8003e-6bfb-4260-b924-3e7c51fe5fdb None admin] RBAC: Authorizing `identity:create_grant(project_id=d0a0755303374019b40301d4352eb9f1, user_id=a3936b76bc4e481e996fb09dc7089c50, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:50.510117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d8003e-6bfb-4260-b924-3e7c51fe5fdb None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:50.523080 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 588/1192] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 10:21:50 2022] PUT /identity/v3/projects/d0a0755303374019b40301d4352eb9f1/users/a3936b76bc4e481e996fb09dc7089c50/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:21:52.650417 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b801a-29d1-4bed-aa02-75b34944ab9e None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:52.651011 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b801a-29d1-4bed-aa02-75b34944ab9e None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:52.651011 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b801a-29d1-4bed-aa02-75b34944ab9e None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:52.652298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 605/1193] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:21:52.658211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401677d-7394-4c07-899e-435d0bbc0837 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:52.658640 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401677d-7394-4c07-899e-435d0bbc0837 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:52.658743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401677d-7394-4c07-899e-435d0bbc0837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:52.676319 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-f401677d-7394-4c07-899e-435d0bbc0837 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:21:52.677486 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-f401677d-7394-4c07-899e-435d0bbc0837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:21:52.740984 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 589/1194] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:21:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:21:53.019987 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b801a-29d1-4bed-aa02-75b34944ab9e None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:53.085806 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:53.118339 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b801a-29d1-4bed-aa02-75b34944ab9e None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:53.151396 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02aa854-cd0a-47d6-83f9-a2e57ffaed8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:53.152003 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02aa854-cd0a-47d6-83f9-a2e57ffaed8c None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:53.152280 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02aa854-cd0a-47d6-83f9-a2e57ffaed8c None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:53.152552 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02aa854-cd0a-47d6-83f9-a2e57ffaed8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:53.162323 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02aa854-cd0a-47d6-83f9-a2e57ffaed8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:21:53.167013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-a02aa854-cd0a-47d6-83f9-a2e57ffaed8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:21:53.167013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:21:53.167013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:21:53.167013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:21:53.167013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:21:53.167013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:21:53.167013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:21:53.167013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:21:53.167013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:21:53.167013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:21:53.167013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:21:53.167013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:21:53.167013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:21:53.167013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:21:53.167013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:21:53.167013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:21:53.168101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:21:53.168101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:21:53.168101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:21:53.168101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:21:53.168101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:21:53.168101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:21:53.168101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:21:53.168101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:21:53.168101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:21:53.168101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:21:53.168101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:21:53.168101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:21:53.168101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:21:53.168101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:21:53.168101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:21:53.168101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:21:53.169165 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:21:53.169165 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:21:53.169165 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:21:53.169165 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:21:53.169165 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:21:53.169165 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:21:53.169165 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:21:53.169165 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:21:53.169165 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:21:53.169165 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:21:53.169165 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:21:53.169165 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:21:53.169165 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 606/1195] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:21:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:21:53.174252 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f401677d-7394-4c07-899e-435d0bbc0837 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:53.253395 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:53.272877 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f401677d-7394-4c07-899e-435d0bbc0837 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:53.305367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afba53d0-8253-4fab-b738-d3c00746c16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:53.305954 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba53d0-8253-4fab-b738-d3c00746c16f None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:53.306257 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba53d0-8253-4fab-b738-d3c00746c16f None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:53.306527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba53d0-8253-4fab-b738-d3c00746c16f None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:53.307172 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afba53d0-8253-4fab-b738-d3c00746c16f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:21:53.307468 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afba53d0-8253-4fab-b738-d3c00746c16f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:53.308815 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afba53d0-8253-4fab-b738-d3c00746c16f None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:53.317024 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 590/1196] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:21:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:53.322912 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02aa854-cd0a-47d6-83f9-a2e57ffaed8c None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:53.426877 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02aa854-cd0a-47d6-83f9-a2e57ffaed8c None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:53.452376 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:53.460824 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93855e-3645-450b-98d0-0b866cbcd13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:53.461439 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93855e-3645-450b-98d0-0b866cbcd13f None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:53.461715 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93855e-3645-450b-98d0-0b866cbcd13f None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:53.461988 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93855e-3645-450b-98d0-0b866cbcd13f None admin] PATH_INFO: `/v3/users/a3936b76bc4e481e996fb09dc7089c50` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:53.463117 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb93855e-3645-450b-98d0-0b866cbcd13f None admin] RBAC: Authorizing `identity:get_user(user_id=a3936b76bc4e481e996fb09dc7089c50)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:53.464556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb93855e-3645-450b-98d0-0b866cbcd13f None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:53.466157 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 607/1197] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Sep 26 10:21:53 2022] GET /identity/v3/users/a3936b76bc4e481e996fb09dc7089c50 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:53.471935 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afba53d0-8253-4fab-b738-d3c00746c16f None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:53.570042 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afba53d0-8253-4fab-b738-d3c00746c16f None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:53.601966 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57076d8-0089-407f-9259-53ff800770e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:53.602590 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57076d8-0089-407f-9259-53ff800770e0 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:53.602866 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57076d8-0089-407f-9259-53ff800770e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:53.603135 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57076d8-0089-407f-9259-53ff800770e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:53.617884 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:53.677610 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57076d8-0089-407f-9259-53ff800770e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:53.679079 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57076d8-0089-407f-9259-53ff800770e0 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:53.772602 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 591/1198] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:21:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 10:21:53.779102 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93855e-3645-450b-98d0-0b866cbcd13f None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:53.876137 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93855e-3645-450b-98d0-0b866cbcd13f None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:53.908715 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37281aaa-0829-41d5-bd6b-53a01e5bb325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:53.909332 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37281aaa-0829-41d5-bd6b-53a01e5bb325 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:53.909611 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37281aaa-0829-41d5-bd6b-53a01e5bb325 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:53.909885 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37281aaa-0829-41d5-bd6b-53a01e5bb325 None admin] PATH_INFO: `/v3/projects/d0a0755303374019b40301d4352eb9f1` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:53.911016 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37281aaa-0829-41d5-bd6b-53a01e5bb325 None admin] RBAC: Authorizing `identity:get_project(project_id=d0a0755303374019b40301d4352eb9f1)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:53.912440 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37281aaa-0829-41d5-bd6b-53a01e5bb325 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:53.914078 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 608/1199] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Sep 26 10:21:53 2022] GET /identity/v3/projects/d0a0755303374019b40301d4352eb9f1 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:21:53.920719 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57076d8-0089-407f-9259-53ff800770e0 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:21:53.964167 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:21:54.023288 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57076d8-0089-407f-9259-53ff800770e0 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:21:54.056716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1aea4d-c490-43bb-a154-081473455a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:21:54.057307 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1aea4d-c490-43bb-a154-081473455a36 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:21:54.057577 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1aea4d-c490-43bb-a154-081473455a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:21:54.057842 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1aea4d-c490-43bb-a154-081473455a36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:21:54.058583 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1aea4d-c490-43bb-a154-081473455a36 None admin] RBAC: Adding query filter params (role.id=532b37d9076c45618cf352aafb85f35e, scope.project.id=d0a0755303374019b40301d4352eb9f1, user.id=a3936b76bc4e481e996fb09dc7089c50) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:21:54.058883 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1aea4d-c490-43bb-a154-081473455a36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:21:54.060316 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1aea4d-c490-43bb-a154-081473455a36 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:21:54.065801 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 592/1200] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:21:53 2022] GET /identity/v3/role_assignments?user.id=a3936b76bc4e481e996fb09dc7089c50&scope.project.id=d0a0755303374019b40301d4352eb9f1&role.id=532b37d9076c45618cf352aafb85f35e => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:22:03.904643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=68 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:22:04.063376 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=46 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:22:49.363058 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0f855-76f5-4a43-9e6e-8324c98ded64 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:22:49.363678 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0f855-76f5-4a43-9e6e-8324c98ded64 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:22:49.363678 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0f855-76f5-4a43-9e6e-8324c98ded64 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:22:49.365118 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 609/1201] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:22:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:22:49.370520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92446a-a454-40e4-bdd9-b5bab166d100 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:22:49.370955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92446a-a454-40e4-bdd9-b5bab166d100 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:22:49.371055 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92446a-a454-40e4-bdd9-b5bab166d100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:22:49.392612 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-8b92446a-a454-40e4-bdd9-b5bab166d100 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:22:49.393964 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-8b92446a-a454-40e4-bdd9-b5bab166d100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:22:49.488822 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 593/1202] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:22:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:22:49.822543 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0f855-76f5-4a43-9e6e-8324c98ded64 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:22:49.963735 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0f855-76f5-4a43-9e6e-8324c98ded64 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:22:50.007784 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409bccb-3903-4ee0-96a9-410f6b57fb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:22:50.008464 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409bccb-3903-4ee0-96a9-410f6b57fb8c None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:22:50.008744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409bccb-3903-4ee0-96a9-410f6b57fb8c None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:22:50.009019 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409bccb-3903-4ee0-96a9-410f6b57fb8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:22:50.023458 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409bccb-3903-4ee0-96a9-410f6b57fb8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:22:50.024950 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409bccb-3903-4ee0-96a9-410f6b57fb8c None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:22:50.045601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-3409bccb-3903-4ee0-96a9-410f6b57fb8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:22:50.045601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:22:50.045601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:22:50.045601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:22:50.045601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:22:50.045601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:22:50.045601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:22:50.045601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:22:50.045601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:22:50.045601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:22:50.045601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:22:50.045601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:22:50.045601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:22:50.045601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:22:50.045601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:22:50.045601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:22:50.046971 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:22:50.046971 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:22:50.046971 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:22:50.046971 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:22:50.046971 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:22:50.046971 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:22:50.046971 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:22:50.046971 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:22:50.046971 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:22:50.046971 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:22:50.046971 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:22:50.046971 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:22:50.046971 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:22:50.046971 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:22:50.046971 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:22:50.046971 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:22:50.046971 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:22:50.047952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:22:50.047952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:22:50.047952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:22:50.047952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 610/1203] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:22:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:22:50.055711 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92446a-a454-40e4-bdd9-b5bab166d100 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:22:50.197938 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92446a-a454-40e4-bdd9-b5bab166d100 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:22:50.244162 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:22:50.249620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f438d16-97f4-43c1-adef-d0e8523b79a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:22:50.250504 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f438d16-97f4-43c1-adef-d0e8523b79a5 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:22:50.250783 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f438d16-97f4-43c1-adef-d0e8523b79a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:22:50.251231 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f438d16-97f4-43c1-adef-d0e8523b79a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:22:50.251886 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f438d16-97f4-43c1-adef-d0e8523b79a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:22:50.252224 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f438d16-97f4-43c1-adef-d0e8523b79a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:22:50.253899 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f438d16-97f4-43c1-adef-d0e8523b79a5 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:22:50.276509 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 594/1204] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:22:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:22:50.279122 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409bccb-3903-4ee0-96a9-410f6b57fb8c None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:22:50.358088 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:22:50.413430 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409bccb-3903-4ee0-96a9-410f6b57fb8c None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:22:50.467793 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7801aa-3b79-49d9-8069-b418268a36f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:22:50.469032 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7801aa-3b79-49d9-8069-b418268a36f1 None admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:22:50.469435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7801aa-3b79-49d9-8069-b418268a36f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:22:50.469678 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7801aa-3b79-49d9-8069-b418268a36f1 None admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:22:50.470653 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7801aa-3b79-49d9-8069-b418268a36f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:22:50.472534 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7801aa-3b79-49d9-8069-b418268a36f1 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:22:50.502670 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 611/1205] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Sep 26 10:22:50 2022] POST /identity/v3/users => generated 251 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:22:52.811815 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd63ab8-fc91-4ebf-947e-1c1c3f7465ae None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:22:52.812371 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd63ab8-fc91-4ebf-947e-1c1c3f7465ae None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:22:52.812563 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd63ab8-fc91-4ebf-947e-1c1c3f7465ae None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:22:52.814212 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 595/1206] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:22:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:22:52.819684 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1d90f-99c6-4402-9f47-880ccbf37cc5 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:22:52.820125 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1d90f-99c6-4402-9f47-880ccbf37cc5 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:22:52.820219 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1d90f-99c6-4402-9f47-880ccbf37cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:22:52.839049 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-e2a1d90f-99c6-4402-9f47-880ccbf37cc5 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:22:52.840289 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-e2a1d90f-99c6-4402-9f47-880ccbf37cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:22:52.912925 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 612/1207] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:22:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:22:53.194492 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd63ab8-fc91-4ebf-947e-1c1c3f7465ae None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:22:53.315085 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd63ab8-fc91-4ebf-947e-1c1c3f7465ae None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:22:53.353609 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0cc96f-d390-481d-b12b-4b21da39ff22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:22:53.354354 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0cc96f-d390-481d-b12b-4b21da39ff22 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:22:53.354650 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0cc96f-d390-481d-b12b-4b21da39ff22 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:22:53.354918 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0cc96f-d390-481d-b12b-4b21da39ff22 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:22:53.367209 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0cc96f-d390-481d-b12b-4b21da39ff22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:22:53.372118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-8b0cc96f-d390-481d-b12b-4b21da39ff22 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:22:53.372118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:22:53.372118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:22:53.372118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:22:53.372118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:22:53.372118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:22:53.372118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:22:53.372118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:22:53.372118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:22:53.372118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:22:53.372118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:22:53.372118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:22:53.372118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:22:53.372118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:22:53.372118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:22:53.372118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:22:53.373133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:22:53.373133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:22:53.373133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:22:53.373133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:22:53.373133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:22:53.373133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:22:53.373133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:22:53.373133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:22:53.373133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:22:53.373133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:22:53.373133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:22:53.373133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:22:53.373133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:22:53.373133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:22:53.373133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:22:53.373133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:22:53.374052 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:22:53.374052 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:22:53.374052 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:22:53.374052 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:22:53.374052 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:22:53.374052 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:22:53.374052 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:22:53.374052 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:22:53.374052 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:22:53.374052 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:22:53.374052 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:22:53.374052 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:22:53.374052 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 596/1208] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 10:22:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:22:53.381247 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1d90f-99c6-4402-9f47-880ccbf37cc5 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:22:53.431456 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:22:53.506314 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1d90f-99c6-4402-9f47-880ccbf37cc5 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:22:53.546126 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c17bf-333b-45a1-8371-c17f36e81655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:22:53.546771 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c17bf-333b-45a1-8371-c17f36e81655 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:22:53.547089 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c17bf-333b-45a1-8371-c17f36e81655 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:22:53.547437 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c17bf-333b-45a1-8371-c17f36e81655 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:22:53.548140 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c17bf-333b-45a1-8371-c17f36e81655 None admin] RBAC: Adding query filter params (name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:22:53.548442 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c17bf-333b-45a1-8371-c17f36e81655 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:22:53.549818 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c17bf-333b-45a1-8371-c17f36e81655 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:22:53.562201 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 613/1209] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 10:22:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:22:53.566259 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0cc96f-d390-481d-b12b-4b21da39ff22 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:22:53.625375 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:22:53.686492 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0cc96f-d390-481d-b12b-4b21da39ff22 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:22:53.721790 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08335e6-e5b4-455e-85ac-8c097cec0d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:22:53.722439 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08335e6-e5b4-455e-85ac-8c097cec0d46 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:22:53.723389 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08335e6-e5b4-455e-85ac-8c097cec0d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:22:53.723389 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08335e6-e5b4-455e-85ac-8c097cec0d46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:22:53.735551 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08335e6-e5b4-455e-85ac-8c097cec0d46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:22:53.737079 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08335e6-e5b4-455e-85ac-8c097cec0d46 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:22:53.751914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-b08335e6-e5b4-455e-85ac-8c097cec0d46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:22:53.751914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:22:53.751914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:22:53.751914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:22:53.751914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:22:53.751914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:22:53.751914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:22:53.751914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:22:53.751914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:22:53.751914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:22:53.751914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:22:53.751914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:22:53.751914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:22:53.751914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:22:53.751914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:22:53.751914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:22:53.753381 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:22:53.753381 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:22:53.753381 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:22:53.753381 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:22:53.753381 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:22:53.753381 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:22:53.753381 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:22:53.753381 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:22:53.753381 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:22:53.753381 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:22:53.753381 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:22:53.753381 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:22:53.753381 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:22:53.753381 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:22:53.753381 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:22:53.753381 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:22:53.753381 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:22:53.754498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:22:53.754498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:22:53.754498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:22:53.754498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 597/1210] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:22:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:22:53.759156 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c17bf-333b-45a1-8371-c17f36e81655 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:22:53.881016 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:22:53.883949 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c17bf-333b-45a1-8371-c17f36e81655 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:22:53.923642 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2dda55-d364-405f-b16c-b5b7a481fd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:22:53.924266 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2dda55-d364-405f-b16c-b5b7a481fd37 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:22:53.924546 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2dda55-d364-405f-b16c-b5b7a481fd37 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:22:53.924822 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2dda55-d364-405f-b16c-b5b7a481fd37 None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:22:53.925476 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2dda55-d364-405f-b16c-b5b7a481fd37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:22:53.925777 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2dda55-d364-405f-b16c-b5b7a481fd37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:22:53.927174 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2dda55-d364-405f-b16c-b5b7a481fd37 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:22:53.941111 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 614/1211] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:22:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:22:53.947346 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08335e6-e5b4-455e-85ac-8c097cec0d46 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:22:54.072354 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08335e6-e5b4-455e-85ac-8c097cec0d46 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:22:54.073951 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:22:54.111434 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f58794-8ab5-4b6c-a3f7-d51e768d9465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:22:54.112281 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f58794-8ab5-4b6c-a3f7-d51e768d9465 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:22:54.112682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f58794-8ab5-4b6c-a3f7-d51e768d9465 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:22:54.113023 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f58794-8ab5-4b6c-a3f7-d51e768d9465 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:22:54.123767 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f58794-8ab5-4b6c-a3f7-d51e768d9465 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:22:54.125697 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f58794-8ab5-4b6c-a3f7-d51e768d9465 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:22:54.141947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-e5f58794-8ab5-4b6c-a3f7-d51e768d9465 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 26 10:22:54.141947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:22:54.141947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:22:54.141947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:22:54.141947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:22:54.141947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:22:54.141947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:22:54.141947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:22:54.141947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:22:54.141947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:22:54.141947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:22:54.141947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:22:54.141947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:22:54.141947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:22:54.141947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:22:54.141947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:22:54.143159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:22:54.143159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:22:54.143159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:22:54.143159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:22:54.143159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:22:54.143159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:22:54.143159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:22:54.143159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:22:54.143159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:22:54.143159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:22:54.143159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:22:54.143159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:22:54.143159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:22:54.143159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:22:54.143159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:22:54.143159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:22:54.143159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:22:54.143159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:22:54.144365 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:22:54.144365 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:22:54.144365 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:22:54.144365 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:22:54.144365 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:22:54.144365 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:22:54.144365 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:22:54.144365 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:22:54.144365 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 26 10:22:54.144365 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:22:54.144365 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 598/1212] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:22:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:22:54.147790 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2dda55-d364-405f-b16c-b5b7a481fd37 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:22:54.267515 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2dda55-d364-405f-b16c-b5b7a481fd37 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:22:54.305103 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d43e75-9c77-4890-a8e5-8ecf72ec24dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:22:54.305691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d43e75-9c77-4890-a8e5-8ecf72ec24dd None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:22:54.305965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d43e75-9c77-4890-a8e5-8ecf72ec24dd None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:22:54.306283 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d43e75-9c77-4890-a8e5-8ecf72ec24dd None admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:22:54.307027 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d43e75-9c77-4890-a8e5-8ecf72ec24dd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:22:54.307305 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d43e75-9c77-4890-a8e5-8ecf72ec24dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:22:54.308764 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d43e75-9c77-4890-a8e5-8ecf72ec24dd None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:22:54.331577 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 615/1213] 10.222.0.49 () {56 vars in 1143 bytes} [Mon Sep 26 10:22:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:22:54.333079 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f58794-8ab5-4b6c-a3f7-d51e768d9465 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:22:54.444491 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f58794-8ab5-4b6c-a3f7-d51e768d9465 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:22:54.483897 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3549c775-6e1c-4e68-b768-4ea2acd7a6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:22:54.484652 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549c775-6e1c-4e68-b768-4ea2acd7a6e4 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:22:54.485062 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549c775-6e1c-4e68-b768-4ea2acd7a6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:22:54.485430 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549c775-6e1c-4e68-b768-4ea2acd7a6e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:22:54.525555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:22:54.572920 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3549c775-6e1c-4e68-b768-4ea2acd7a6e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:22:54.574919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3549c775-6e1c-4e68-b768-4ea2acd7a6e4 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:22:54.699230 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 599/1214] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:22:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 10:22:54.705483 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d43e75-9c77-4890-a8e5-8ecf72ec24dd None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:22:54.729414 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:22:54.831591 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d43e75-9c77-4890-a8e5-8ecf72ec24dd None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:22:54.879551 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774396b7-25a0-4424-85bc-5148d69d1263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:22:54.880746 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774396b7-25a0-4424-85bc-5148d69d1263 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:22:54.881023 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774396b7-25a0-4424-85bc-5148d69d1263 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:22:54.881564 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774396b7-25a0-4424-85bc-5148d69d1263 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:22:54.894245 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774396b7-25a0-4424-85bc-5148d69d1263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:22:54.896249 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774396b7-25a0-4424-85bc-5148d69d1263 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:22:54.912293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-774396b7-25a0-4424-85bc-5148d69d1263 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:22:54.912293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:22:54.912293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:22:54.912293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:22:54.912293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:22:54.912293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:22:54.912293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:22:54.912293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:22:54.912293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:22:54.912293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:22:54.912293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:22:54.912293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:22:54.912293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:22:54.912293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:22:54.912293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:22:54.912293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:22:54.913770 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:22:54.913770 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:22:54.913770 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:22:54.913770 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:22:54.913770 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:22:54.913770 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:22:54.913770 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:22:54.913770 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:22:54.913770 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:22:54.913770 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:22:54.913770 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:22:54.913770 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:22:54.913770 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:22:54.913770 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:22:54.913770 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:22:54.913770 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:22:54.913770 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:22:54.914722 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:22:54.914722 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:22:54.914722 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:22:54.914722 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 616/1215] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:22:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:22:54.919133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3549c775-6e1c-4e68-b768-4ea2acd7a6e4 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:22:54.992562 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:22:55.041233 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3549c775-6e1c-4e68-b768-4ea2acd7a6e4 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:22:55.083731 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05bbe2b-53c0-4163-ba78-7cd57d926e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:22:55.084727 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bbe2b-53c0-4163-ba78-7cd57d926e29 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:22:55.085346 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bbe2b-53c0-4163-ba78-7cd57d926e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:22:55.085710 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bbe2b-53c0-4163-ba78-7cd57d926e29 None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:22:55.086486 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05bbe2b-53c0-4163-ba78-7cd57d926e29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:22:55.086880 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05bbe2b-53c0-4163-ba78-7cd57d926e29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:22:55.088387 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05bbe2b-53c0-4163-ba78-7cd57d926e29 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:22:55.101680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 600/1216] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:22:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:22:55.107422 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774396b7-25a0-4424-85bc-5148d69d1263 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:22:55.187654 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:22:55.220781 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774396b7-25a0-4424-85bc-5148d69d1263 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:22:55.263257 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f7da2-2d0c-4a0e-8e70-955459c217f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:22:55.263257 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f7da2-2d0c-4a0e-8e70-955459c217f6 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:22:55.263257 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f7da2-2d0c-4a0e-8e70-955459c217f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:22:55.263257 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f7da2-2d0c-4a0e-8e70-955459c217f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:22:55.269108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84f7da2-2d0c-4a0e-8e70-955459c217f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:22:55.270715 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84f7da2-2d0c-4a0e-8e70-955459c217f6 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:22:55.285823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-d84f7da2-2d0c-4a0e-8e70-955459c217f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:22:55.285823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:22:55.285823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:22:55.285823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:22:55.285823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:22:55.285823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:22:55.285823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:22:55.285823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:22:55.285823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:22:55.285823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:22:55.285823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:22:55.285823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:22:55.285823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:22:55.285823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:22:55.285823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:22:55.286981 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:22:55.286981 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:22:55.286981 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:22:55.286981 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:22:55.286981 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:22:55.286981 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:22:55.286981 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:22:55.286981 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:22:55.286981 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:22:55.286981 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:22:55.286981 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:22:55.286981 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:22:55.286981 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:22:55.286981 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:22:55.286981 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:22:55.286981 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:22:55.286981 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:22:55.291252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:22:55.291252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:22:55.291252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:22:55.291252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:22:55.291252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:22:55.291252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:22:55.291252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:22:55.291252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:22:55.291252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 617/1217] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:22:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:22:55.292937 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05bbe2b-53c0-4163-ba78-7cd57d926e29 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:22:55.426352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05bbe2b-53c0-4163-ba78-7cd57d926e29 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:22:55.459788 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:22:55.473178 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b54f5-c1ec-43d1-82d1-3533ffdd68d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:22:55.473886 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b54f5-c1ec-43d1-82d1-3533ffdd68d9 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:22:55.474231 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b54f5-c1ec-43d1-82d1-3533ffdd68d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:22:55.474536 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b54f5-c1ec-43d1-82d1-3533ffdd68d9 None admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:22:55.475226 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117b54f5-c1ec-43d1-82d1-3533ffdd68d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:22:55.475524 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117b54f5-c1ec-43d1-82d1-3533ffdd68d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:22:55.476934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117b54f5-c1ec-43d1-82d1-3533ffdd68d9 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:22:55.488699 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 601/1218] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:22:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:22:55.495009 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f7da2-2d0c-4a0e-8e70-955459c217f6 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:22:55.602963 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f7da2-2d0c-4a0e-8e70-955459c217f6 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:22:55.634990 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:22:55.641854 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa9b31-2ee9-49cd-a4cd-38dfb175045d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:22:55.642667 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa9b31-2ee9-49cd-a4cd-38dfb175045d None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:22:55.643111 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa9b31-2ee9-49cd-a4cd-38dfb175045d None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:22:55.643557 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa9b31-2ee9-49cd-a4cd-38dfb175045d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:22:55.644469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fa9b31-2ee9-49cd-a4cd-38dfb175045d None admin] RBAC: Adding query filter params (role.id=d36e58344b604bcda1cee49065e8497e, scope.project.id=cff3a6011f4c4893bddffdad3d69f3df, user.id=84b02e7351914cabb67462454b328d05) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:22:55.644915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fa9b31-2ee9-49cd-a4cd-38dfb175045d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:22:55.646473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fa9b31-2ee9-49cd-a4cd-38dfb175045d None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:22:55.652429 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 618/1219] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:22:55 2022] GET /identity/v3/role_assignments?user.id=84b02e7351914cabb67462454b328d05&scope.project.id=cff3a6011f4c4893bddffdad3d69f3df&role.id=d36e58344b604bcda1cee49065e8497e => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:22:58.053115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160a215-dd99-40ba-a947-2b92b58fef85 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:22:58.053666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160a215-dd99-40ba-a947-2b92b58fef85 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:22:58.053666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160a215-dd99-40ba-a947-2b92b58fef85 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:22:58.055024 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 602/1220] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:22:58.062132 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247db2f4-5223-4a61-b15b-0685f65a3851 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:22:58.062132 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247db2f4-5223-4a61-b15b-0685f65a3851 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:22:58.062132 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247db2f4-5223-4a61-b15b-0685f65a3851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:22:58.081761 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-247db2f4-5223-4a61-b15b-0685f65a3851 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:22:58.083148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-247db2f4-5223-4a61-b15b-0685f65a3851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:22:58.157468 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 619/1221] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:22:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:22:58.471041 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a160a215-dd99-40ba-a947-2b92b58fef85 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:22:58.581019 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a160a215-dd99-40ba-a947-2b92b58fef85 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:22:58.618531 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6590203-a0bb-44a7-b1f7-0815e7a0f562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:22:58.619218 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6590203-a0bb-44a7-b1f7-0815e7a0f562 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:22:58.619493 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6590203-a0bb-44a7-b1f7-0815e7a0f562 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:22:58.619758 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6590203-a0bb-44a7-b1f7-0815e7a0f562 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:22:58.629884 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6590203-a0bb-44a7-b1f7-0815e7a0f562 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:22:58.634655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-e6590203-a0bb-44a7-b1f7-0815e7a0f562 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:22:58.634655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:22:58.634655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:22:58.634655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:22:58.634655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:22:58.634655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:22:58.634655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:22:58.634655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:22:58.634655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:22:58.634655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:22:58.634655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:22:58.634655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:22:58.634655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:22:58.634655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:22:58.634655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:22:58.634655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:22:58.637906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:22:58.637906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:22:58.637906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:22:58.637906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:22:58.637906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:22:58.637906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:22:58.637906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:22:58.637906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:22:58.637906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:22:58.637906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:22:58.637906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:22:58.637906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:22:58.637906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:22:58.637906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:22:58.637906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:22:58.637906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:22:58.639164 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:22:58.639164 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:22:58.639164 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:22:58.639164 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:22:58.639164 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:22:58.639164 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:22:58.639164 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:22:58.639164 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:22:58.639164 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:22:58.639164 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:22:58.639164 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:22:58.639164 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:22:58.639164 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 603/1222] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 10:22:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:22:58.642384 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247db2f4-5223-4a61-b15b-0685f65a3851 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:22:58.753401 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247db2f4-5223-4a61-b15b-0685f65a3851 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:22:58.782369 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:22:58.796322 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a645c40-2f6a-47a0-83f0-f0cedde994d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:22:58.796993 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a645c40-2f6a-47a0-83f0-f0cedde994d5 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:22:58.797270 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a645c40-2f6a-47a0-83f0-f0cedde994d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:22:58.797553 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a645c40-2f6a-47a0-83f0-f0cedde994d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:22:58.798350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a645c40-2f6a-47a0-83f0-f0cedde994d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:22:58.798649 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a645c40-2f6a-47a0-83f0-f0cedde994d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:22:58.800131 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a645c40-2f6a-47a0-83f0-f0cedde994d5 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:22:58.809631 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 620/1223] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 10:22:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:22:58.815506 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6590203-a0bb-44a7-b1f7-0815e7a0f562 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:22:58.864673 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:22:58.930805 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6590203-a0bb-44a7-b1f7-0815e7a0f562 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:22:58.973595 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4e591-ea3f-43b1-bdc5-5cad4b6e6c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:22:58.974374 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4e591-ea3f-43b1-bdc5-5cad4b6e6c38 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:22:58.974782 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4e591-ea3f-43b1-bdc5-5cad4b6e6c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:22:58.975186 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4e591-ea3f-43b1-bdc5-5cad4b6e6c38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:22:58.987848 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4e591-ea3f-43b1-bdc5-5cad4b6e6c38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:22:58.989451 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4e591-ea3f-43b1-bdc5-5cad4b6e6c38 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:22:59.007863 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-21a4e591-ea3f-43b1-bdc5-5cad4b6e6c38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:22:59.007863 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:22:59.007863 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:22:59.007863 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:22:59.007863 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:22:59.007863 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:22:59.007863 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:22:59.007863 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:22:59.007863 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:22:59.007863 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:22:59.007863 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:22:59.007863 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:22:59.007863 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:22:59.007863 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:22:59.007863 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:22:59.007863 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:22:59.008984 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:22:59.008984 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:22:59.008984 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:22:59.008984 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:22:59.008984 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:22:59.008984 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:22:59.008984 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:22:59.008984 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:22:59.008984 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:22:59.008984 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:22:59.008984 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:22:59.008984 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:22:59.008984 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:22:59.008984 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:22:59.008984 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:22:59.008984 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:22:59.008984 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:22:59.010853 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:22:59.010853 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:22:59.010853 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:22:59.010853 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 604/1224] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:22:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:22:59.016261 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a645c40-2f6a-47a0-83f0-f0cedde994d5 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:22:59.131532 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a645c40-2f6a-47a0-83f0-f0cedde994d5 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:22:59.170912 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a8b3c-ffab-47ca-912c-9c039b4973f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:22:59.171524 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a8b3c-ffab-47ca-912c-9c039b4973f1 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:22:59.171934 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a8b3c-ffab-47ca-912c-9c039b4973f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:22:59.172341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a8b3c-ffab-47ca-912c-9c039b4973f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:22:59.173114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a8b3c-ffab-47ca-912c-9c039b4973f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:22:59.173483 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a8b3c-ffab-47ca-912c-9c039b4973f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:22:59.175181 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a8b3c-ffab-47ca-912c-9c039b4973f1 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:22:59.188907 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 621/1225] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:22:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:22:59.194855 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4e591-ea3f-43b1-bdc5-5cad4b6e6c38 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:22:59.306382 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:22:59.315442 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4e591-ea3f-43b1-bdc5-5cad4b6e6c38 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:22:59.352361 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3ae71b-4158-41ee-97ea-9cf09f6abe35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:22:59.352988 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ae71b-4158-41ee-97ea-9cf09f6abe35 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:22:59.353312 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ae71b-4158-41ee-97ea-9cf09f6abe35 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:22:59.353568 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ae71b-4158-41ee-97ea-9cf09f6abe35 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:22:59.362485 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3ae71b-4158-41ee-97ea-9cf09f6abe35 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:22:59.364013 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3ae71b-4158-41ee-97ea-9cf09f6abe35 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:22:59.376932 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-cb3ae71b-4158-41ee-97ea-9cf09f6abe35 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 26 10:22:59.376932 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:22:59.376932 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:22:59.376932 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:22:59.376932 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:22:59.376932 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:22:59.376932 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:22:59.376932 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:22:59.376932 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:22:59.376932 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:22:59.376932 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:22:59.376932 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:22:59.376932 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:22:59.376932 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:22:59.376932 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:22:59.376932 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:22:59.382772 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:22:59.382772 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:22:59.382772 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:22:59.382772 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:22:59.382772 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:22:59.382772 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:22:59.382772 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:22:59.382772 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:22:59.382772 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:22:59.382772 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:22:59.382772 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:22:59.382772 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:22:59.382772 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:22:59.382772 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:22:59.382772 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:22:59.382772 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:22:59.382772 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:22:59.382772 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:22:59.383996 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:22:59.383996 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:22:59.383996 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:22:59.383996 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:22:59.383996 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:22:59.383996 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:22:59.383996 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:22:59.383996 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:22:59.383996 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 26 10:22:59.383996 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:22:59.383996 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 605/1226] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:22:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:22:59.384723 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a8b3c-ffab-47ca-912c-9c039b4973f1 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:22:59.437972 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:22:59.498359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a8b3c-ffab-47ca-912c-9c039b4973f1 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:22:59.538488 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e79624-aa71-4d0d-91b5-243801ae6564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:22:59.539135 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e79624-aa71-4d0d-91b5-243801ae6564 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:22:59.539408 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e79624-aa71-4d0d-91b5-243801ae6564 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:22:59.539676 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e79624-aa71-4d0d-91b5-243801ae6564 None admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:22:59.540395 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e79624-aa71-4d0d-91b5-243801ae6564 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:22:59.540694 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e79624-aa71-4d0d-91b5-243801ae6564 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:22:59.542124 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e79624-aa71-4d0d-91b5-243801ae6564 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:22:59.561581 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 622/1227] 10.222.0.49 () {56 vars in 1143 bytes} [Mon Sep 26 10:22:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:22:59.567346 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3ae71b-4158-41ee-97ea-9cf09f6abe35 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:22:59.685727 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3ae71b-4158-41ee-97ea-9cf09f6abe35 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:22:59.720880 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257215f6-f519-463c-90e3-5b62dc78a5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:22:59.721496 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257215f6-f519-463c-90e3-5b62dc78a5fa None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:22:59.721768 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257215f6-f519-463c-90e3-5b62dc78a5fa None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:22:59.722055 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257215f6-f519-463c-90e3-5b62dc78a5fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:22:59.732899 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257215f6-f519-463c-90e3-5b62dc78a5fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:22:59.734572 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257215f6-f519-463c-90e3-5b62dc78a5fa None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:22:59.750116 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-257215f6-f519-463c-90e3-5b62dc78a5fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:22:59.750116 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:22:59.750116 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:22:59.750116 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:22:59.750116 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:22:59.750116 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:22:59.750116 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:22:59.750116 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:22:59.750116 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:22:59.750116 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:22:59.750116 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:22:59.750116 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:22:59.750116 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:22:59.750116 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:22:59.750116 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:22:59.750116 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:22:59.751210 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:22:59.751210 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:22:59.751210 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:22:59.751210 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:22:59.751210 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:22:59.751210 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:22:59.751210 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:22:59.751210 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:22:59.751210 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:22:59.751210 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:22:59.751210 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:22:59.751210 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:22:59.751210 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:22:59.751210 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:22:59.751210 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:22:59.751210 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:22:59.751210 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:22:59.752185 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:22:59.752185 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:22:59.752185 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:22:59.752360 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 606/1228] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:22:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:22:59.756964 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e79624-aa71-4d0d-91b5-243801ae6564 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:22:59.871365 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e79624-aa71-4d0d-91b5-243801ae6564 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:22:59.873949 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:22:59.909982 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09da76f-82d3-44af-8312-015c98a32f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:22:59.910714 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09da76f-82d3-44af-8312-015c98a32f0b None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:22:59.911132 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09da76f-82d3-44af-8312-015c98a32f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:22:59.911407 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09da76f-82d3-44af-8312-015c98a32f0b None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:22:59.912211 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09da76f-82d3-44af-8312-015c98a32f0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:22:59.912571 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09da76f-82d3-44af-8312-015c98a32f0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:22:59.914124 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09da76f-82d3-44af-8312-015c98a32f0b None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:22:59.927884 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 623/1229] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:22:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:22:59.933715 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257215f6-f519-463c-90e3-5b62dc78a5fa None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:22:59.943671 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:00.054040 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257215f6-f519-463c-90e3-5b62dc78a5fa None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:00.090003 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d10303-405b-4bf3-add8-961b7eb3b36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:00.091331 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d10303-405b-4bf3-add8-961b7eb3b36e None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:00.091645 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d10303-405b-4bf3-add8-961b7eb3b36e None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:00.091914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d10303-405b-4bf3-add8-961b7eb3b36e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:00.099155 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d10303-405b-4bf3-add8-961b7eb3b36e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:00.100580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d10303-405b-4bf3-add8-961b7eb3b36e None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:00.112916 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-68d10303-405b-4bf3-add8-961b7eb3b36e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:23:00.112916 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:23:00.112916 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:23:00.112916 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:23:00.112916 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:23:00.112916 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:23:00.112916 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:23:00.112916 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:23:00.112916 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:23:00.112916 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:23:00.112916 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:23:00.112916 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:23:00.112916 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:23:00.112916 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:23:00.112916 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:23:00.113944 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:23:00.113944 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:23:00.113944 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:23:00.113944 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:23:00.113944 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:23:00.113944 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:23:00.113944 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:23:00.113944 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:23:00.113944 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:23:00.113944 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:23:00.113944 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:23:00.113944 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:23:00.113944 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:23:00.113944 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:23:00.113944 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:23:00.113944 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:23:00.113944 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:23:00.116146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:23:00.116146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:23:00.116146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:23:00.116146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:23:00.116146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:23:00.116146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:23:00.116146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:23:00.116146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:23:00.116146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 607/1230] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:22:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:23:00.120049 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09da76f-82d3-44af-8312-015c98a32f0b None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:00.229933 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09da76f-82d3-44af-8312-015c98a32f0b None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:00.266337 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab311a9-2fa3-4cf3-8f87-216f94d209d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:00.266949 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab311a9-2fa3-4cf3-8f87-216f94d209d3 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:00.267234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab311a9-2fa3-4cf3-8f87-216f94d209d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:00.267508 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab311a9-2fa3-4cf3-8f87-216f94d209d3 None admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:00.268195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab311a9-2fa3-4cf3-8f87-216f94d209d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:23:00.268525 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab311a9-2fa3-4cf3-8f87-216f94d209d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:00.269938 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab311a9-2fa3-4cf3-8f87-216f94d209d3 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:00.283598 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 624/1231] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:23:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:23:00.288535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d10303-405b-4bf3-add8-961b7eb3b36e None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:00.365346 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:00.408042 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d10303-405b-4bf3-add8-961b7eb3b36e None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:00.447293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77584309-5eff-4b9f-8865-9bc0bdfbe7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:00.448035 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77584309-5eff-4b9f-8865-9bc0bdfbe7b5 None admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:00.448453 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77584309-5eff-4b9f-8865-9bc0bdfbe7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:00.448675 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77584309-5eff-4b9f-8865-9bc0bdfbe7b5 None admin] PATH_INFO: `/v3/projects/cff3a6011f4c4893bddffdad3d69f3df/users/84b02e7351914cabb67462454b328d05/roles/d36e58344b604bcda1cee49065e8497e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:00.469869 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77584309-5eff-4b9f-8865-9bc0bdfbe7b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=cff3a6011f4c4893bddffdad3d69f3df, user_id=84b02e7351914cabb67462454b328d05, role_id=d36e58344b604bcda1cee49065e8497e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:00.471907 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77584309-5eff-4b9f-8865-9bc0bdfbe7b5 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:00.487941 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 608/1232] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 10:23:00 2022] PUT /identity/v3/projects/cff3a6011f4c4893bddffdad3d69f3df/users/84b02e7351914cabb67462454b328d05/roles/d36e58344b604bcda1cee49065e8497e => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:23:02.745468 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2eefae-5f07-478f-b9e1-7543e2b67259 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:02.746047 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2eefae-5f07-478f-b9e1-7543e2b67259 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:02.746047 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2eefae-5f07-478f-b9e1-7543e2b67259 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:02.747585 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 625/1233] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:23:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:23:02.753438 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883047e-da14-41b5-8eff-867f59a9eaf1 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:02.753833 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883047e-da14-41b5-8eff-867f59a9eaf1 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:02.754074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883047e-da14-41b5-8eff-867f59a9eaf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:02.774554 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-2883047e-da14-41b5-8eff-867f59a9eaf1 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:23:02.776317 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-2883047e-da14-41b5-8eff-867f59a9eaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:23:02.857874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 609/1234] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:23:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:23:03.124105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2eefae-5f07-478f-b9e1-7543e2b67259 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:03.149860 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:03.229323 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2eefae-5f07-478f-b9e1-7543e2b67259 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:03.264144 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57cc857-1328-4410-b443-234461250aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:03.264777 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57cc857-1328-4410-b443-234461250aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:03.265057 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57cc857-1328-4410-b443-234461250aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:03.265333 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57cc857-1328-4410-b443-234461250aa7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:03.277079 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57cc857-1328-4410-b443-234461250aa7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:23:03.282743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-a57cc857-1328-4410-b443-234461250aa7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:23:03.282743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:23:03.282743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:23:03.282743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:23:03.282743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:23:03.282743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:23:03.282743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:23:03.282743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:23:03.282743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:23:03.282743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:23:03.282743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:23:03.282743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:23:03.282743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:23:03.282743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:23:03.282743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:23:03.282743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:23:03.283851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:23:03.283851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:23:03.283851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:23:03.283851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:23:03.283851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:23:03.283851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:23:03.283851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:23:03.283851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:23:03.283851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:23:03.283851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:23:03.283851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:23:03.283851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:23:03.283851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:23:03.283851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:23:03.283851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:23:03.283851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:23:03.284715 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:23:03.284715 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:23:03.284715 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:23:03.284715 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:23:03.284715 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:23:03.284715 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:23:03.284715 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:23:03.284715 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:23:03.284715 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:23:03.284715 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:23:03.284715 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 10:23:03.284715 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:23:03.284715 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 626/1235] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Sep 26 10:23:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 10:23:03.290096 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2883047e-da14-41b5-8eff-867f59a9eaf1 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:03.337877 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:03.395580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2883047e-da14-41b5-8eff-867f59a9eaf1 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:03.436598 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea257ae-deb3-47d3-98f6-ae0fbea604ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:03.437360 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea257ae-deb3-47d3-98f6-ae0fbea604ec None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:03.437756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea257ae-deb3-47d3-98f6-ae0fbea604ec None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:03.438130 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea257ae-deb3-47d3-98f6-ae0fbea604ec None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:03.438867 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea257ae-deb3-47d3-98f6-ae0fbea604ec None admin] RBAC: Adding query filter params (name=service) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:23:03.439297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea257ae-deb3-47d3-98f6-ae0fbea604ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:03.440790 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea257ae-deb3-47d3-98f6-ae0fbea604ec None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:03.451889 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 610/1236] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Sep 26 10:23:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:23:03.459693 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57cc857-1328-4410-b443-234461250aa7 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:03.556214 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:03.573728 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57cc857-1328-4410-b443-234461250aa7 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:03.616757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf50c1a-b0eb-41dd-a7c2-970f57fa3cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:03.617513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf50c1a-b0eb-41dd-a7c2-970f57fa3cfe None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:03.617789 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf50c1a-b0eb-41dd-a7c2-970f57fa3cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:03.618102 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf50c1a-b0eb-41dd-a7c2-970f57fa3cfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:03.630570 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf50c1a-b0eb-41dd-a7c2-970f57fa3cfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:03.632017 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf50c1a-b0eb-41dd-a7c2-970f57fa3cfe None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:03.651179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-8cf50c1a-b0eb-41dd-a7c2-970f57fa3cfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:23:03.651179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:23:03.651179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:23:03.651179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:23:03.651179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:23:03.651179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:23:03.651179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:23:03.651179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:23:03.651179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:23:03.651179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:23:03.651179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:23:03.651179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:23:03.651179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:23:03.651179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:23:03.651179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:23:03.651179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:23:03.652387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:23:03.652387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:23:03.652387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:23:03.652387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:23:03.652387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:23:03.652387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:23:03.652387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:23:03.652387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:23:03.652387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:23:03.652387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:23:03.652387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:23:03.652387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:23:03.652387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:23:03.652387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:23:03.652387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:23:03.652387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:23:03.652387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:23:03.653658 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:23:03.653658 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:23:03.653658 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:23:03.653658 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 627/1237] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:23:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:23:03.658709 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea257ae-deb3-47d3-98f6-ae0fbea604ec None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:03.773051 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea257ae-deb3-47d3-98f6-ae0fbea604ec None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:03.782627 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:03.813569 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57f44a-35d8-48a5-9f4e-b0fc8d7591e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:03.814208 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57f44a-35d8-48a5-9f4e-b0fc8d7591e2 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:03.814500 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57f44a-35d8-48a5-9f4e-b0fc8d7591e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:03.814793 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57f44a-35d8-48a5-9f4e-b0fc8d7591e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:03.815557 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e57f44a-35d8-48a5-9f4e-b0fc8d7591e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:23:03.815899 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e57f44a-35d8-48a5-9f4e-b0fc8d7591e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:03.817492 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e57f44a-35d8-48a5-9f4e-b0fc8d7591e2 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:03.829890 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 611/1238] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:23:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:23:03.835765 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf50c1a-b0eb-41dd-a7c2-970f57fa3cfe None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:03.941050 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf50c1a-b0eb-41dd-a7c2-970f57fa3cfe None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:03.974520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5a1fb-7b71-45ad-b79e-818ca4b1bba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:03.975148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5a1fb-7b71-45ad-b79e-818ca4b1bba8 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:03.975428 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5a1fb-7b71-45ad-b79e-818ca4b1bba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:03.975701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5a1fb-7b71-45ad-b79e-818ca4b1bba8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:03.978702 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:03.985435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce5a1fb-7b71-45ad-b79e-818ca4b1bba8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:03.987373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce5a1fb-7b71-45ad-b79e-818ca4b1bba8 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:04.003015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-4ce5a1fb-7b71-45ad-b79e-818ca4b1bba8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 26 10:23:04.003015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:23:04.003015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:23:04.003015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:23:04.003015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:23:04.003015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:23:04.003015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:23:04.003015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:23:04.003015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:23:04.003015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:23:04.003015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:23:04.003015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:23:04.003015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:23:04.003015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:23:04.003015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:23:04.003015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:23:04.004100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:23:04.004100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:23:04.004100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:23:04.004100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:23:04.004100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:23:04.004100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:23:04.004100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:23:04.004100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:23:04.004100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:23:04.004100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:23:04.004100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:23:04.004100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:23:04.004100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:23:04.004100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:23:04.004100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:23:04.004100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:23:04.004100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:23:04.004100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:23:04.005070 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:23:04.005070 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:23:04.005070 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:23:04.005070 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:23:04.005070 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:23:04.005070 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:23:04.005070 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:23:04.005070 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:23:04.005070 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 26 10:23:04.005070 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:23:04.005070 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 628/1239] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:23:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:23:04.010477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57f44a-35d8-48a5-9f4e-b0fc8d7591e2 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:04.128816 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57f44a-35d8-48a5-9f4e-b0fc8d7591e2 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:04.168666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec2abb-be43-4fdf-a700-82622790bf22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:04.169636 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec2abb-be43-4fdf-a700-82622790bf22 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:04.170143 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec2abb-be43-4fdf-a700-82622790bf22 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:04.170624 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec2abb-be43-4fdf-a700-82622790bf22 None admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:04.171830 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec2abb-be43-4fdf-a700-82622790bf22 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:23:04.172238 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec2abb-be43-4fdf-a700-82622790bf22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:04.174004 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec2abb-be43-4fdf-a700-82622790bf22 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:04.203361 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 612/1240] 10.222.0.49 () {56 vars in 1143 bytes} [Mon Sep 26 10:23:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:23:04.204466 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5a1fb-7b71-45ad-b79e-818ca4b1bba8 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:04.332094 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5a1fb-7b71-45ad-b79e-818ca4b1bba8 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:04.370381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614b4a8-cf45-4aa8-85ec-8901a2a9a40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:04.371016 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614b4a8-cf45-4aa8-85ec-8901a2a9a40f None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:04.371294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614b4a8-cf45-4aa8-85ec-8901a2a9a40f None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:04.371597 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614b4a8-cf45-4aa8-85ec-8901a2a9a40f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:04.447690 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:04.456777 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b614b4a8-cf45-4aa8-85ec-8901a2a9a40f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:04.458937 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b614b4a8-cf45-4aa8-85ec-8901a2a9a40f None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:04.572806 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 629/1241] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:23:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 10:23:04.579129 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec2abb-be43-4fdf-a700-82622790bf22 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:04.609765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:04.691054 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec2abb-be43-4fdf-a700-82622790bf22 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:04.733919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3db448-31f4-4ef8-aa77-374c8cdce00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:04.734617 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3db448-31f4-4ef8-aa77-374c8cdce00a None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:04.734926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3db448-31f4-4ef8-aa77-374c8cdce00a None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:04.735245 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3db448-31f4-4ef8-aa77-374c8cdce00a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:04.747941 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3db448-31f4-4ef8-aa77-374c8cdce00a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:04.749661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3db448-31f4-4ef8-aa77-374c8cdce00a None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:04.767127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-ff3db448-31f4-4ef8-aa77-374c8cdce00a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:23:04.767127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:23:04.767127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:23:04.767127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:23:04.767127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:23:04.767127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:23:04.767127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:23:04.767127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:23:04.767127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:23:04.767127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:23:04.767127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:23:04.767127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:23:04.767127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:23:04.767127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:23:04.767127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:23:04.767127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:23:04.768297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:23:04.768297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:23:04.768297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:23:04.768297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:23:04.768297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:23:04.768297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:23:04.768297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:23:04.768297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:23:04.768297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:23:04.768297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:23:04.768297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:23:04.768297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:23:04.768297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:23:04.768297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:23:04.768297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:23:04.768297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:23:04.768297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:23:04.769543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:23:04.769543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:23:04.769543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:23:04.769543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 613/1242] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:23:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:23:04.774441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614b4a8-cf45-4aa8-85ec-8901a2a9a40f None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:04.881845 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:04.889737 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614b4a8-cf45-4aa8-85ec-8901a2a9a40f None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:04.926149 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff226ff-8510-4611-a6df-38c4b3ae14e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:04.926835 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff226ff-8510-4611-a6df-38c4b3ae14e2 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:04.927181 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff226ff-8510-4611-a6df-38c4b3ae14e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:04.927465 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff226ff-8510-4611-a6df-38c4b3ae14e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:04.928115 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff226ff-8510-4611-a6df-38c4b3ae14e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:23:04.928429 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff226ff-8510-4611-a6df-38c4b3ae14e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:04.929790 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff226ff-8510-4611-a6df-38c4b3ae14e2 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:04.941933 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 630/1243] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:23:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:23:04.947685 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3db448-31f4-4ef8-aa77-374c8cdce00a None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:05.036605 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:05.059777 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3db448-31f4-4ef8-aa77-374c8cdce00a None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:05.105127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa95413-0e80-4330-a8fa-7e7ccb9e379c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:05.105744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa95413-0e80-4330-a8fa-7e7ccb9e379c None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:05.106041 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa95413-0e80-4330-a8fa-7e7ccb9e379c None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:05.106342 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa95413-0e80-4330-a8fa-7e7ccb9e379c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:05.113270 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa95413-0e80-4330-a8fa-7e7ccb9e379c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:05.114966 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa95413-0e80-4330-a8fa-7e7ccb9e379c None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:05.127211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-faa95413-0e80-4330-a8fa-7e7ccb9e379c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:23:05.127211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:23:05.127211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:23:05.127211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:23:05.127211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:23:05.127211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:23:05.127211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:23:05.127211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:23:05.127211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:23:05.127211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:23:05.127211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:23:05.127211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:23:05.127211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:23:05.127211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:23:05.127211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:23:05.128180 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:23:05.128180 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:23:05.128180 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:23:05.128180 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:23:05.128180 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:23:05.128180 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:23:05.128180 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:23:05.128180 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:23:05.128180 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:23:05.128180 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:23:05.128180 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:23:05.128180 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:23:05.128180 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:23:05.128180 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:23:05.128180 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:23:05.128180 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:23:05.128180 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:23:05.129123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:23:05.129123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:23:05.129123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:23:05.129123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:23:05.129123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:23:05.129123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:23:05.129123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:23:05.129123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:23:05.129123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 614/1244] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:23:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:23:05.134639 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff226ff-8510-4611-a6df-38c4b3ae14e2 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:05.242217 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff226ff-8510-4611-a6df-38c4b3ae14e2 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:05.279102 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7e6f0-9358-46db-ba28-38958f8a9884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:05.279650 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7e6f0-9358-46db-ba28-38958f8a9884 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:05.279940 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7e6f0-9358-46db-ba28-38958f8a9884 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:05.280209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7e6f0-9358-46db-ba28-38958f8a9884 None admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:05.280898 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc7e6f0-9358-46db-ba28-38958f8a9884 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:23:05.281209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc7e6f0-9358-46db-ba28-38958f8a9884 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:05.282667 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc7e6f0-9358-46db-ba28-38958f8a9884 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:05.297276 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 631/1245] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:23:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:23:05.303371 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa95413-0e80-4330-a8fa-7e7ccb9e379c None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:05.411165 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa95413-0e80-4330-a8fa-7e7ccb9e379c None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:05.452363 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6607ddac-1678-4eba-ab0e-fbf83b9d5af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:05.453054 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607ddac-1678-4eba-ab0e-fbf83b9d5af6 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:05.453333 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607ddac-1678-4eba-ab0e-fbf83b9d5af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:05.453601 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607ddac-1678-4eba-ab0e-fbf83b9d5af6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:05.454458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6607ddac-1678-4eba-ab0e-fbf83b9d5af6 None admin] RBAC: Adding query filter params (role.id=d36e58344b604bcda1cee49065e8497e, scope.project.id=cff3a6011f4c4893bddffdad3d69f3df, user.id=84b02e7351914cabb67462454b328d05) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:23:05.454807 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6607ddac-1678-4eba-ab0e-fbf83b9d5af6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:05.456403 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6607ddac-1678-4eba-ab0e-fbf83b9d5af6 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:05.463350 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 615/1246] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:23:05 2022] GET /identity/v3/role_assignments?user.id=84b02e7351914cabb67462454b328d05&scope.project.id=cff3a6011f4c4893bddffdad3d69f3df&role.id=d36e58344b604bcda1cee49065e8497e => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:23:07.745042 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85265b34-89fe-4d25-bb69-710b536e289d None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:07.745662 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85265b34-89fe-4d25-bb69-710b536e289d None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:07.745662 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85265b34-89fe-4d25-bb69-710b536e289d None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:07.746992 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 632/1247] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:23:07.752286 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f2900-1e35-4d8f-85fa-5371374e5876 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:07.752812 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f2900-1e35-4d8f-85fa-5371374e5876 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:07.752812 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f2900-1e35-4d8f-85fa-5371374e5876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:07.760174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:07.774505 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-879f2900-1e35-4d8f-85fa-5371374e5876 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:23:07.775704 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-879f2900-1e35-4d8f-85fa-5371374e5876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:23:07.854481 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 616/1248] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:23:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:23:08.122964 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85265b34-89fe-4d25-bb69-710b536e289d None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:08.146245 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:08.241420 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85265b34-89fe-4d25-bb69-710b536e289d None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:08.280307 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d42c175-499a-4b99-9620-9fc207e2ea71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:08.280938 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42c175-499a-4b99-9620-9fc207e2ea71 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:08.281228 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42c175-499a-4b99-9620-9fc207e2ea71 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:08.281497 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42c175-499a-4b99-9620-9fc207e2ea71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:08.293702 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d42c175-499a-4b99-9620-9fc207e2ea71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:23:08.298533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-6d42c175-499a-4b99-9620-9fc207e2ea71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:23:08.298533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:23:08.298533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:23:08.298533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:23:08.298533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:23:08.298533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:23:08.298533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:23:08.298533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:23:08.298533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:23:08.298533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:23:08.298533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:23:08.298533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:23:08.298533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:23:08.298533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:23:08.298533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:23:08.298533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:23:08.299850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:23:08.299850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:23:08.299850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:23:08.299850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:23:08.299850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:23:08.299850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:23:08.299850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:23:08.299850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:23:08.299850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:23:08.299850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:23:08.299850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:23:08.299850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:23:08.299850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:23:08.299850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:23:08.299850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:23:08.299850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:23:08.301191 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:23:08.301191 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:23:08.301191 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:23:08.301191 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:23:08.301191 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:23:08.301191 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:23:08.301191 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:23:08.301191 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:23:08.301191 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:23:08.301191 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:23:08.301191 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:23:08.301191 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:23:08.301191 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 633/1249] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:23:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:23:08.306959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879f2900-1e35-4d8f-85fa-5371374e5876 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:08.419267 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879f2900-1e35-4d8f-85fa-5371374e5876 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:08.462963 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:08.469854 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734d3a5-a0af-4336-9819-1b78423987eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:08.470547 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734d3a5-a0af-4336-9819-1b78423987eb None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:08.470841 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734d3a5-a0af-4336-9819-1b78423987eb None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:08.471251 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734d3a5-a0af-4336-9819-1b78423987eb None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:08.472021 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734d3a5-a0af-4336-9819-1b78423987eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:23:08.472357 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734d3a5-a0af-4336-9819-1b78423987eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:08.474297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734d3a5-a0af-4336-9819-1b78423987eb None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:08.493497 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 617/1250] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:23:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:23:08.495613 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d42c175-499a-4b99-9620-9fc207e2ea71 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:08.577557 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:08.602328 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d42c175-499a-4b99-9620-9fc207e2ea71 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:08.638609 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6bfb6-d9f9-41e7-8936-8487695cf2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:08.639258 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6bfb6-d9f9-41e7-8936-8487695cf2bc None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:08.639537 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6bfb6-d9f9-41e7-8936-8487695cf2bc None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:08.639808 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6bfb6-d9f9-41e7-8936-8487695cf2bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:08.650850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e6bfb6-d9f9-41e7-8936-8487695cf2bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:08.652829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e6bfb6-d9f9-41e7-8936-8487695cf2bc None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:08.669065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-d0e6bfb6-d9f9-41e7-8936-8487695cf2bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:23:08.669065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:23:08.669065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:23:08.669065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:23:08.669065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:23:08.669065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:23:08.669065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:23:08.669065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:23:08.669065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:23:08.669065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:23:08.669065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:23:08.669065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:23:08.669065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:23:08.669065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:23:08.669065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:23:08.669065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:23:08.670374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:23:08.670374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:23:08.670374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:23:08.670374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:23:08.670374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:23:08.670374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:23:08.670374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:23:08.670374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:23:08.670374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:23:08.670374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:23:08.670374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:23:08.670374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:23:08.670374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:23:08.670374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:23:08.670374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:23:08.670374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:23:08.670374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:23:08.671491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:23:08.671491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:23:08.671491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:23:08.671491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 634/1251] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:23:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:23:08.676326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734d3a5-a0af-4336-9819-1b78423987eb None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:08.781075 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734d3a5-a0af-4336-9819-1b78423987eb None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:08.821493 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6225e-902f-4db7-a655-582cda39e639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:08.822146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6225e-902f-4db7-a655-582cda39e639 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:08.822425 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6225e-902f-4db7-a655-582cda39e639 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:08.822699 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6225e-902f-4db7-a655-582cda39e639 None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:08.823353 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b6225e-902f-4db7-a655-582cda39e639 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:23:08.824161 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b6225e-902f-4db7-a655-582cda39e639 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:08.825102 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b6225e-902f-4db7-a655-582cda39e639 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:08.838079 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 618/1252] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:23:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:23:08.844407 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6bfb6-d9f9-41e7-8936-8487695cf2bc None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:08.952862 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6bfb6-d9f9-41e7-8936-8487695cf2bc None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:08.979474 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:08.988053 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ff4c0-6ab9-4c2f-9cf2-ba3fd0f635e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:08.988681 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ff4c0-6ab9-4c2f-9cf2-ba3fd0f635e1 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:08.988961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ff4c0-6ab9-4c2f-9cf2-ba3fd0f635e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:08.989236 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ff4c0-6ab9-4c2f-9cf2-ba3fd0f635e1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:08.996590 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ff4c0-6ab9-4c2f-9cf2-ba3fd0f635e1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:08.998081 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ff4c0-6ab9-4c2f-9cf2-ba3fd0f635e1 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:09.008974 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-7d3ff4c0-6ab9-4c2f-9cf2-ba3fd0f635e1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 26 10:23:09.008974 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:23:09.008974 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:23:09.008974 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:23:09.008974 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:23:09.008974 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:23:09.008974 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:23:09.008974 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:23:09.008974 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:23:09.008974 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:23:09.008974 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:23:09.008974 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:23:09.008974 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:23:09.008974 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:23:09.008974 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:23:09.008974 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:23:09.010134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:23:09.010134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:23:09.010134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:23:09.010134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:23:09.010134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:23:09.010134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:23:09.010134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:23:09.010134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:23:09.010134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:23:09.010134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:23:09.010134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:23:09.010134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:23:09.010134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:23:09.010134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:23:09.010134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:23:09.010134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:23:09.010134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:23:09.010134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:23:09.011386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:23:09.011386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:23:09.011386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:23:09.011386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:23:09.011386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:23:09.011386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:23:09.011386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:23:09.011386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:23:09.011386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 26 10:23:09.011386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:23:09.011386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 635/1253] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:23:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:23:09.016491 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6225e-902f-4db7-a655-582cda39e639 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:09.070997 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:09.115450 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6225e-902f-4db7-a655-582cda39e639 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:09.149248 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24be672-a543-4e27-bae0-683487ec72ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:09.149846 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24be672-a543-4e27-bae0-683487ec72ca None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:09.150161 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24be672-a543-4e27-bae0-683487ec72ca None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:09.150438 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24be672-a543-4e27-bae0-683487ec72ca None admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:09.151161 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24be672-a543-4e27-bae0-683487ec72ca None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:23:09.151463 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24be672-a543-4e27-bae0-683487ec72ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:09.153500 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24be672-a543-4e27-bae0-683487ec72ca None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:09.169901 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 619/1254] 10.222.0.49 () {56 vars in 1143 bytes} [Mon Sep 26 10:23:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:23:09.175644 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ff4c0-6ab9-4c2f-9cf2-ba3fd0f635e1 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:09.273668 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ff4c0-6ab9-4c2f-9cf2-ba3fd0f635e1 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:09.306036 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0925cb-7908-44c5-b711-ecfa22ca7fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:09.306638 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0925cb-7908-44c5-b711-ecfa22ca7fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:09.306918 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0925cb-7908-44c5-b711-ecfa22ca7fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:09.307188 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0925cb-7908-44c5-b711-ecfa22ca7fa3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:09.360054 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:09.373107 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0925cb-7908-44c5-b711-ecfa22ca7fa3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:09.374566 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0925cb-7908-44c5-b711-ecfa22ca7fa3 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:09.480953 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 636/1255] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:23:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 10:23:09.486798 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24be672-a543-4e27-bae0-683487ec72ca None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:09.583734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24be672-a543-4e27-bae0-683487ec72ca None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:09.596499 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:09.616796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52473e21-f0ff-40fc-ba3a-6ca8d012c49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:09.617411 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52473e21-f0ff-40fc-ba3a-6ca8d012c49a None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:09.617685 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52473e21-f0ff-40fc-ba3a-6ca8d012c49a None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:09.617950 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52473e21-f0ff-40fc-ba3a-6ca8d012c49a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:09.628082 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52473e21-f0ff-40fc-ba3a-6ca8d012c49a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:09.629504 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52473e21-f0ff-40fc-ba3a-6ca8d012c49a None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:09.644082 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-52473e21-f0ff-40fc-ba3a-6ca8d012c49a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:23:09.644082 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:23:09.644082 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:23:09.644082 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:23:09.644082 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:23:09.644082 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:23:09.644082 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:23:09.644082 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:23:09.644082 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:23:09.644082 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:23:09.644082 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:23:09.644082 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:23:09.644082 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:23:09.644082 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:23:09.644082 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:23:09.644082 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:23:09.645207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:23:09.645207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:23:09.645207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:23:09.645207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:23:09.645207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:23:09.645207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:23:09.645207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:23:09.645207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:23:09.645207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:23:09.645207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:23:09.645207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:23:09.645207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:23:09.645207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:23:09.645207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:23:09.645207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:23:09.645207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:23:09.645207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:23:09.646401 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:23:09.646401 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:23:09.646401 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:23:09.646401 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 620/1256] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:23:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:23:09.650922 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0925cb-7908-44c5-b711-ecfa22ca7fa3 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:09.734399 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:09.746958 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0925cb-7908-44c5-b711-ecfa22ca7fa3 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:09.779575 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc5792-215e-4085-a098-95a8d7276bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:09.780161 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc5792-215e-4085-a098-95a8d7276bfe None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:09.780440 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc5792-215e-4085-a098-95a8d7276bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:09.780709 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc5792-215e-4085-a098-95a8d7276bfe None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:09.781336 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc5792-215e-4085-a098-95a8d7276bfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:23:09.781633 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc5792-215e-4085-a098-95a8d7276bfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:09.783004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc5792-215e-4085-a098-95a8d7276bfe None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:09.796515 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 637/1257] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:23:09.802050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52473e21-f0ff-40fc-ba3a-6ca8d012c49a None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:09.897990 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52473e21-f0ff-40fc-ba3a-6ca8d012c49a None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:09.931067 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254bc55e-2ce0-4be9-8af5-e634dc643608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:09.931667 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254bc55e-2ce0-4be9-8af5-e634dc643608 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:09.931938 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254bc55e-2ce0-4be9-8af5-e634dc643608 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:09.932202 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254bc55e-2ce0-4be9-8af5-e634dc643608 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:09.939181 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254bc55e-2ce0-4be9-8af5-e634dc643608 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:09.940596 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254bc55e-2ce0-4be9-8af5-e634dc643608 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:09.951533 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-254bc55e-2ce0-4be9-8af5-e634dc643608 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:23:09.951533 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:23:09.951533 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:23:09.951533 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:23:09.951533 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:23:09.951533 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:23:09.951533 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:23:09.951533 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:23:09.951533 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:23:09.951533 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:23:09.951533 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:23:09.951533 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:23:09.951533 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:23:09.951533 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:23:09.951533 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:23:09.952604 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:23:09.952604 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:23:09.952604 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:23:09.952604 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:23:09.952604 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:23:09.952604 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:23:09.952604 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:23:09.952604 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:23:09.952604 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:23:09.952604 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:23:09.952604 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:23:09.952604 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:23:09.952604 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:23:09.952604 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:23:09.952604 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:23:09.952604 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:23:09.952604 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:23:09.953527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:23:09.953527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:23:09.953527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:23:09.953527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:23:09.953527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:23:09.953527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:23:09.953527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:23:09.953527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:23:09.953527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 621/1258] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:23:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:23:09.958525 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc5792-215e-4085-a098-95a8d7276bfe None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:10.055595 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc5792-215e-4085-a098-95a8d7276bfe None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:10.088320 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60110735-3c15-4e65-b4de-e1d99daea7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:10.088910 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60110735-3c15-4e65-b4de-e1d99daea7a2 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:10.089187 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60110735-3c15-4e65-b4de-e1d99daea7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:10.089459 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60110735-3c15-4e65-b4de-e1d99daea7a2 None admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:10.090168 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60110735-3c15-4e65-b4de-e1d99daea7a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:23:10.090474 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60110735-3c15-4e65-b4de-e1d99daea7a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:10.091896 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60110735-3c15-4e65-b4de-e1d99daea7a2 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:10.103409 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 638/1259] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:23:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:23:10.109912 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254bc55e-2ce0-4be9-8af5-e634dc643608 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:10.135102 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:10.206725 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254bc55e-2ce0-4be9-8af5-e634dc643608 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:10.239367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a881f7-31eb-45b2-9a4c-a219cfb25894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:10.239939 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a881f7-31eb-45b2-9a4c-a219cfb25894 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:10.240211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a881f7-31eb-45b2-9a4c-a219cfb25894 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:10.240479 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a881f7-31eb-45b2-9a4c-a219cfb25894 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:10.241185 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a881f7-31eb-45b2-9a4c-a219cfb25894 None admin] RBAC: Adding query filter params (role.id=532b37d9076c45618cf352aafb85f35e, scope.project.id=cff3a6011f4c4893bddffdad3d69f3df, user.id=84b02e7351914cabb67462454b328d05) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:23:10.241484 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a881f7-31eb-45b2-9a4c-a219cfb25894 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:10.242910 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a881f7-31eb-45b2-9a4c-a219cfb25894 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:10.248115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 622/1260] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:23:10 2022] GET /identity/v3/role_assignments?user.id=84b02e7351914cabb67462454b328d05&scope.project.id=cff3a6011f4c4893bddffdad3d69f3df&role.id=532b37d9076c45618cf352aafb85f35e => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:23:12.376989 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59914974-2fa5-448d-989a-79ca65594c22 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:12.377521 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59914974-2fa5-448d-989a-79ca65594c22 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:12.377521 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59914974-2fa5-448d-989a-79ca65594c22 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:12.378899 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 639/1261] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:23:12.384016 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d45ee-2e62-4915-bd3f-887d04c2b464 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:12.384413 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d45ee-2e62-4915-bd3f-887d04c2b464 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:12.384548 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d45ee-2e62-4915-bd3f-887d04c2b464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:12.402958 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-a50d45ee-2e62-4915-bd3f-887d04c2b464 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:23:12.404123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-a50d45ee-2e62-4915-bd3f-887d04c2b464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:23:12.479358 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 623/1262] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:23:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:23:12.760294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59914974-2fa5-448d-989a-79ca65594c22 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:12.772147 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:12.878378 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59914974-2fa5-448d-989a-79ca65594c22 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:12.912436 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d8549-6dad-4b56-bee5-9ac16ede7287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:12.913062 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d8549-6dad-4b56-bee5-9ac16ede7287 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:12.913336 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d8549-6dad-4b56-bee5-9ac16ede7287 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:12.913604 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d8549-6dad-4b56-bee5-9ac16ede7287 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:12.924144 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86d8549-6dad-4b56-bee5-9ac16ede7287 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:23:12.928845 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-f86d8549-6dad-4b56-bee5-9ac16ede7287 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:23:12.928845 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:23:12.928845 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:23:12.928845 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:23:12.928845 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:23:12.928845 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:23:12.928845 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:23:12.928845 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:23:12.928845 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:23:12.928845 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:23:12.928845 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:23:12.928845 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:23:12.928845 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:23:12.928845 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:23:12.928845 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:23:12.928845 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:23:12.929764 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:23:12.929764 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:23:12.929764 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:23:12.929764 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:23:12.929764 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:23:12.929764 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:23:12.929764 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:23:12.929764 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:23:12.929764 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:23:12.929764 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:23:12.929764 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:23:12.929764 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:23:12.929764 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:23:12.929764 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:23:12.929764 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:23:12.929764 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:23:12.930866 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:23:12.930866 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:23:12.930866 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:23:12.930866 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:23:12.930866 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:23:12.930866 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:23:12.930866 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:23:12.930866 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:23:12.930866 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:23:12.930866 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:23:12.930866 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:23:12.930866 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:23:12.930866 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 640/1263] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:23:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:23:12.936859 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d45ee-2e62-4915-bd3f-887d04c2b464 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:12.955202 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:13.041334 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d45ee-2e62-4915-bd3f-887d04c2b464 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:13.075212 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0273fefb-1cf1-4636-9b04-4981eb770944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:13.075810 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273fefb-1cf1-4636-9b04-4981eb770944 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:13.076087 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273fefb-1cf1-4636-9b04-4981eb770944 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:13.076368 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273fefb-1cf1-4636-9b04-4981eb770944 None admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:13.077015 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0273fefb-1cf1-4636-9b04-4981eb770944 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:23:13.077309 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0273fefb-1cf1-4636-9b04-4981eb770944 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:13.078708 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0273fefb-1cf1-4636-9b04-4981eb770944 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:13.086929 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 624/1264] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:23:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:23:13.092850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d8549-6dad-4b56-bee5-9ac16ede7287 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:13.159015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:13.191695 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d8549-6dad-4b56-bee5-9ac16ede7287 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:13.224530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b95bb-107c-452b-ac47-288c63d26a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:13.225138 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b95bb-107c-452b-ac47-288c63d26a53 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:13.225415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b95bb-107c-452b-ac47-288c63d26a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:13.225687 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b95bb-107c-452b-ac47-288c63d26a53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:13.236001 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b95bb-107c-452b-ac47-288c63d26a53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:13.237429 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b95bb-107c-452b-ac47-288c63d26a53 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:13.251535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-aa2b95bb-107c-452b-ac47-288c63d26a53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:23:13.251535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:23:13.251535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:23:13.251535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:23:13.251535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:23:13.251535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:23:13.251535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:23:13.251535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:23:13.251535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:23:13.251535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:23:13.251535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:23:13.251535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:23:13.251535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:23:13.251535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:23:13.251535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:23:13.251535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:23:13.252608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:23:13.252608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:23:13.252608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:23:13.252608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:23:13.252608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:23:13.252608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:23:13.252608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:23:13.252608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:23:13.252608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:23:13.252608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:23:13.252608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:23:13.252608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:23:13.252608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:23:13.252608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:23:13.252608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:23:13.252608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:23:13.252608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:23:13.253507 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:23:13.253507 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:23:13.253507 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:23:13.253507 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 641/1265] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:23:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:23:13.258523 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0273fefb-1cf1-4636-9b04-4981eb770944 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:13.336858 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:13.359262 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0273fefb-1cf1-4636-9b04-4981eb770944 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:13.391662 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4eefc6-d3e5-48ee-bbcd-f9ac5eed32da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:13.392261 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4eefc6-d3e5-48ee-bbcd-f9ac5eed32da None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:13.392519 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4eefc6-d3e5-48ee-bbcd-f9ac5eed32da None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:13.392787 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4eefc6-d3e5-48ee-bbcd-f9ac5eed32da None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:13.393414 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4eefc6-d3e5-48ee-bbcd-f9ac5eed32da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:23:13.393710 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4eefc6-d3e5-48ee-bbcd-f9ac5eed32da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:13.395087 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4eefc6-d3e5-48ee-bbcd-f9ac5eed32da None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:13.406928 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 625/1266] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:23:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:23:13.412615 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b95bb-107c-452b-ac47-288c63d26a53 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:13.513106 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b95bb-107c-452b-ac47-288c63d26a53 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:13.531816 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:13.546124 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b177d8-9605-4a7d-8419-ee4539907570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:13.546747 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b177d8-9605-4a7d-8419-ee4539907570 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:13.547024 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b177d8-9605-4a7d-8419-ee4539907570 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:13.547296 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b177d8-9605-4a7d-8419-ee4539907570 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:13.554420 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b177d8-9605-4a7d-8419-ee4539907570 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:13.555847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b177d8-9605-4a7d-8419-ee4539907570 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:13.567266 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-e1b177d8-9605-4a7d-8419-ee4539907570 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 26 10:23:13.567266 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:23:13.567266 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:23:13.567266 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:23:13.567266 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:23:13.567266 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:23:13.567266 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:23:13.567266 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:23:13.567266 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:23:13.567266 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:23:13.567266 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:23:13.567266 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:23:13.567266 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:23:13.567266 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:23:13.567266 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:23:13.567266 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:23:13.568197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:23:13.568197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:23:13.568197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:23:13.568197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:23:13.568197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:23:13.568197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:23:13.568197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:23:13.568197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:23:13.568197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:23:13.568197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:23:13.568197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:23:13.568197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:23:13.568197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:23:13.568197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:23:13.568197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:23:13.568197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:23:13.568197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:23:13.568197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:23:13.569230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:23:13.569230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:23:13.569230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:23:13.569230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:23:13.569230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:23:13.569230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:23:13.569230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:23:13.569230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:23:13.569230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 26 10:23:13.569230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:23:13.569230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 642/1267] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:23:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:23:13.574149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4eefc6-d3e5-48ee-bbcd-f9ac5eed32da None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:13.669835 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4eefc6-d3e5-48ee-bbcd-f9ac5eed32da None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:13.702785 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97523e-3d0c-4d1e-954e-d0da53b0293e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:13.703369 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97523e-3d0c-4d1e-954e-d0da53b0293e None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:13.703641 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97523e-3d0c-4d1e-954e-d0da53b0293e None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:13.703908 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97523e-3d0c-4d1e-954e-d0da53b0293e None admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:13.704618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d97523e-3d0c-4d1e-954e-d0da53b0293e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:23:13.704915 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d97523e-3d0c-4d1e-954e-d0da53b0293e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:13.706342 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d97523e-3d0c-4d1e-954e-d0da53b0293e None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:13.714968 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:13.723249 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 626/1268] 10.222.0.49 () {56 vars in 1143 bytes} [Mon Sep 26 10:23:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:23:13.728979 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b177d8-9605-4a7d-8419-ee4539907570 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:13.826856 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b177d8-9605-4a7d-8419-ee4539907570 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:13.859112 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e048a2-1820-48ab-a5f1-e69052def243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:13.859716 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e048a2-1820-48ab-a5f1-e69052def243 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:13.859990 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e048a2-1820-48ab-a5f1-e69052def243 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:13.860258 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e048a2-1820-48ab-a5f1-e69052def243 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:13.870353 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e048a2-1820-48ab-a5f1-e69052def243 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:13.872879 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e048a2-1820-48ab-a5f1-e69052def243 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:13.887007 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-12e048a2-1820-48ab-a5f1-e69052def243 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:23:13.887007 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:23:13.887007 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:23:13.887007 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:23:13.887007 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:23:13.887007 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:23:13.887007 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:23:13.887007 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:23:13.887007 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:23:13.887007 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:23:13.887007 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:23:13.887007 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:23:13.887007 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:23:13.887007 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:23:13.887007 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:23:13.887007 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:23:13.888093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:23:13.888093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:23:13.888093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:23:13.888093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:23:13.888093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:23:13.888093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:23:13.888093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:23:13.888093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:23:13.888093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:23:13.888093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:23:13.888093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:23:13.888093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:23:13.888093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:23:13.888093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:23:13.888093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:23:13.888093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:23:13.888093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:23:13.889357 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:23:13.889357 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:23:13.889357 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:23:13.889357 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 643/1269] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:23:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:23:13.893804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97523e-3d0c-4d1e-954e-d0da53b0293e None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:13.990731 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97523e-3d0c-4d1e-954e-d0da53b0293e None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:14.022767 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f70e9-2911-48ca-82f4-f48942177e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:14.023358 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f70e9-2911-48ca-82f4-f48942177e57 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:14.023634 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f70e9-2911-48ca-82f4-f48942177e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:14.023904 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f70e9-2911-48ca-82f4-f48942177e57 None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:14.024534 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f70e9-2911-48ca-82f4-f48942177e57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:23:14.024832 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f70e9-2911-48ca-82f4-f48942177e57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:14.026226 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f70e9-2911-48ca-82f4-f48942177e57 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:14.037489 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 627/1270] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:23:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:23:14.043596 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e048a2-1820-48ab-a5f1-e69052def243 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:14.073824 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:14.141860 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e048a2-1820-48ab-a5f1-e69052def243 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:14.173615 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c751b8e9-f4fa-41cb-b627-0d723a8fd937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:14.174249 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751b8e9-f4fa-41cb-b627-0d723a8fd937 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:14.174526 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751b8e9-f4fa-41cb-b627-0d723a8fd937 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:14.174792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751b8e9-f4fa-41cb-b627-0d723a8fd937 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:14.181139 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c751b8e9-f4fa-41cb-b627-0d723a8fd937 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:14.182587 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c751b8e9-f4fa-41cb-b627-0d723a8fd937 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:14.194554 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-c751b8e9-f4fa-41cb-b627-0d723a8fd937 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:23:14.194554 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:23:14.194554 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:23:14.194554 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:23:14.194554 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:23:14.194554 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:23:14.194554 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:23:14.194554 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:23:14.194554 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:23:14.194554 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:23:14.194554 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:23:14.194554 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:23:14.194554 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:23:14.194554 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:23:14.194554 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:23:14.195481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:23:14.195481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:23:14.195481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:23:14.195481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:23:14.195481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:23:14.195481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:23:14.195481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:23:14.195481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:23:14.195481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:23:14.195481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:23:14.195481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:23:14.195481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:23:14.195481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:23:14.195481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:23:14.195481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:23:14.195481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:23:14.195481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:23:14.196387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:23:14.196387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:23:14.196387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:23:14.196387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:23:14.196387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:23:14.196387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:23:14.196387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:23:14.196387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:23:14.196387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 644/1271] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:23:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:23:14.201881 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f70e9-2911-48ca-82f4-f48942177e57 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:14.257765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:14.298861 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f70e9-2911-48ca-82f4-f48942177e57 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:14.331369 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd4b07-43a0-4930-878d-4cc4574a5ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:14.331965 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd4b07-43a0-4930-878d-4cc4574a5ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:14.332244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd4b07-43a0-4930-878d-4cc4574a5ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:14.332515 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd4b07-43a0-4930-878d-4cc4574a5ef8 None admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:14.333197 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd4b07-43a0-4930-878d-4cc4574a5ef8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:23:14.333499 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd4b07-43a0-4930-878d-4cc4574a5ef8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:14.334946 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd4b07-43a0-4930-878d-4cc4574a5ef8 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:14.347565 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 628/1272] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:23:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:23:14.353364 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c751b8e9-f4fa-41cb-b627-0d723a8fd937 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:14.443801 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:14.454945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c751b8e9-f4fa-41cb-b627-0d723a8fd937 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:14.488807 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375b65a-b9d7-44b7-841e-53764a70303d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:14.489450 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375b65a-b9d7-44b7-841e-53764a70303d None admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:14.489724 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375b65a-b9d7-44b7-841e-53764a70303d None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:14.489995 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375b65a-b9d7-44b7-841e-53764a70303d None admin] PATH_INFO: `/v3/projects/cff3a6011f4c4893bddffdad3d69f3df/users/84b02e7351914cabb67462454b328d05/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:14.491917 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a375b65a-b9d7-44b7-841e-53764a70303d None admin] RBAC: Authorizing `identity:create_grant(project_id=cff3a6011f4c4893bddffdad3d69f3df, user_id=84b02e7351914cabb67462454b328d05, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:14.493317 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a375b65a-b9d7-44b7-841e-53764a70303d None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:14.504926 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 645/1273] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Sep 26 10:23:14 2022] PUT /identity/v3/projects/cff3a6011f4c4893bddffdad3d69f3df/users/84b02e7351914cabb67462454b328d05/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:23:16.614862 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b15656-5aeb-4f30-bdb3-1039643b1b95 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:16.615385 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b15656-5aeb-4f30-bdb3-1039643b1b95 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:16.615504 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b15656-5aeb-4f30-bdb3-1039643b1b95 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:16.616744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 629/1274] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:23:16.622643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117db53-bee4-44ac-97e8-8c52a13081d3 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:16.623042 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117db53-bee4-44ac-97e8-8c52a13081d3 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:16.623182 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117db53-bee4-44ac-97e8-8c52a13081d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:16.640815 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-2117db53-bee4-44ac-97e8-8c52a13081d3 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:23:16.641984 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-2117db53-bee4-44ac-97e8-8c52a13081d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:23:16.706479 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 646/1275] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:23:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:23:16.973006 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b15656-5aeb-4f30-bdb3-1039643b1b95 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:17.070773 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b15656-5aeb-4f30-bdb3-1039643b1b95 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:17.087945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:17.104220 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d0e9b-b0b8-4e22-9807-2060f922e9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:17.104833 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d0e9b-b0b8-4e22-9807-2060f922e9bd None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:17.105105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d0e9b-b0b8-4e22-9807-2060f922e9bd None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:17.105406 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d0e9b-b0b8-4e22-9807-2060f922e9bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:17.115088 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d0e9b-b0b8-4e22-9807-2060f922e9bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 10:23:17.119923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-cc5d0e9b-b0b8-4e22-9807-2060f922e9bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:23:17.119923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:23:17.119923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:23:17.119923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:23:17.119923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:23:17.119923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:23:17.119923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:23:17.119923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:23:17.119923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:23:17.119923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:23:17.119923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:23:17.119923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:23:17.119923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 10:23:17.119923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 10:23:17.119923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 10:23:17.119923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 10:23:17.121477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 10:23:17.121477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 10:23:17.121477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 10:23:17.121477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 10:23:17.121477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:23:17.121477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:23:17.121477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:23:17.121477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:23:17.121477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:23:17.121477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:23:17.121477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:23:17.121477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:23:17.121477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:23:17.121477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:23:17.121477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:23:17.121477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:23:17.123166 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:23:17.123166 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:23:17.123166 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:23:17.123166 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:23:17.123166 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 10:23:17.123166 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 10:23:17.123166 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 10:23:17.123166 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 10:23:17.123166 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 10:23:17.123166 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 10:23:17.123166 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 10:23:17.123166 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:23:17.123166 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 630/1276] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Sep 26 10:23:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:23:17.127475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117db53-bee4-44ac-97e8-8c52a13081d3 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:17.204598 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:17.223138 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117db53-bee4-44ac-97e8-8c52a13081d3 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:17.255494 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc26765-4473-47ed-89d1-57b3fe2a4414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:17.256089 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc26765-4473-47ed-89d1-57b3fe2a4414 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:17.256365 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc26765-4473-47ed-89d1-57b3fe2a4414 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:17.256635 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc26765-4473-47ed-89d1-57b3fe2a4414 None admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:17.257278 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc26765-4473-47ed-89d1-57b3fe2a4414 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:23:17.257575 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc26765-4473-47ed-89d1-57b3fe2a4414 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:17.259008 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc26765-4473-47ed-89d1-57b3fe2a4414 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:17.267306 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 647/1277] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Sep 26 10:23:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:23:17.273078 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d0e9b-b0b8-4e22-9807-2060f922e9bd None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:17.369124 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d0e9b-b0b8-4e22-9807-2060f922e9bd None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:17.401587 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59690758-c63a-4a7e-9c08-0440cc33eb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:17.402213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59690758-c63a-4a7e-9c08-0440cc33eb06 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:17.402630 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59690758-c63a-4a7e-9c08-0440cc33eb06 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:17.402951 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59690758-c63a-4a7e-9c08-0440cc33eb06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:17.412439 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59690758-c63a-4a7e-9c08-0440cc33eb06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:17.413847 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59690758-c63a-4a7e-9c08-0440cc33eb06 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:17.430073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-59690758-c63a-4a7e-9c08-0440cc33eb06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:23:17.430073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:23:17.430073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:23:17.430073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:23:17.430073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:23:17.430073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:23:17.430073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:23:17.430073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:23:17.430073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:23:17.430073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:23:17.430073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:23:17.430073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:23:17.430073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:23:17.430073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:23:17.430073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:23:17.430073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:23:17.431321 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:23:17.431321 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:23:17.431321 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:23:17.431321 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:23:17.431321 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:23:17.431321 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:23:17.431321 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:23:17.431321 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:23:17.431321 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:23:17.431321 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:23:17.431321 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:23:17.431321 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:23:17.431321 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:23:17.431321 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:23:17.431321 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:23:17.431321 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:23:17.431321 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:23:17.432280 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:23:17.432280 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:23:17.432280 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:23:17.432280 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 631/1278] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:23:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:23:17.438374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc26765-4473-47ed-89d1-57b3fe2a4414 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:17.542238 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc26765-4473-47ed-89d1-57b3fe2a4414 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:17.575123 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6efc9d-2577-4765-b20a-4bb68a9c0a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:17.575720 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6efc9d-2577-4765-b20a-4bb68a9c0a92 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:17.576001 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6efc9d-2577-4765-b20a-4bb68a9c0a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:17.576277 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6efc9d-2577-4765-b20a-4bb68a9c0a92 None admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:17.576909 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6efc9d-2577-4765-b20a-4bb68a9c0a92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:23:17.577211 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6efc9d-2577-4765-b20a-4bb68a9c0a92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:17.578629 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6efc9d-2577-4765-b20a-4bb68a9c0a92 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:17.590263 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 648/1279] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:23:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:23:17.595943 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59690758-c63a-4a7e-9c08-0440cc33eb06 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:17.626839 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:17.694362 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59690758-c63a-4a7e-9c08-0440cc33eb06 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:17.727076 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08389713-8f93-4a05-8088-4c8449f5a0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:17.727684 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08389713-8f93-4a05-8088-4c8449f5a0d3 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:17.727959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08389713-8f93-4a05-8088-4c8449f5a0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:17.728227 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08389713-8f93-4a05-8088-4c8449f5a0d3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:17.735977 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08389713-8f93-4a05-8088-4c8449f5a0d3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:17.737388 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08389713-8f93-4a05-8088-4c8449f5a0d3 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:17.748319 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-08389713-8f93-4a05-8088-4c8449f5a0d3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 26 10:23:17.748319 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:23:17.748319 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:23:17.748319 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:23:17.748319 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:23:17.748319 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:23:17.748319 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:23:17.748319 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:23:17.748319 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:23:17.748319 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:23:17.748319 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:23:17.748319 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:23:17.748319 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 10:23:17.748319 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 10:23:17.748319 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 10:23:17.748319 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 10:23:17.749417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:23:17.749417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:23:17.749417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 10:23:17.749417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:23:17.749417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 10:23:17.749417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 10:23:17.749417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:23:17.749417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:23:17.749417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:23:17.749417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:23:17.749417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:23:17.749417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:23:17.749417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:23:17.749417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:23:17.749417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:23:17.749417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:23:17.749417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:23:17.749417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:23:17.750416 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:23:17.750416 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:23:17.750416 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 10:23:17.750416 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 10:23:17.750416 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 10:23:17.750416 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 10:23:17.750416 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 10:23:17.750416 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 10:23:17.750416 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 26 10:23:17.750416 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:23:17.750416 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 632/1280] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:23:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:23:17.755181 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6efc9d-2577-4765-b20a-4bb68a9c0a92 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:17.758953 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:17.852964 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6efc9d-2577-4765-b20a-4bb68a9c0a92 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:17.886098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239c673-b328-4099-b37b-894d18aa310f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:17.886698 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239c673-b328-4099-b37b-894d18aa310f None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:17.886980 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239c673-b328-4099-b37b-894d18aa310f None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:17.887252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239c673-b328-4099-b37b-894d18aa310f None admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:17.887967 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239c673-b328-4099-b37b-894d18aa310f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:23:17.888273 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239c673-b328-4099-b37b-894d18aa310f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:17.889685 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239c673-b328-4099-b37b-894d18aa310f None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:17.907256 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 649/1281] 10.222.0.49 () {56 vars in 1143 bytes} [Mon Sep 26 10:23:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:23:17.912928 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08389713-8f93-4a05-8088-4c8449f5a0d3 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:18.002107 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:18.011442 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08389713-8f93-4a05-8088-4c8449f5a0d3 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:18.044196 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc2a99-dc09-444b-bfd7-c302177d98f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:18.044787 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc2a99-dc09-444b-bfd7-c302177d98f7 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:18.045064 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc2a99-dc09-444b-bfd7-c302177d98f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:18.045339 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc2a99-dc09-444b-bfd7-c302177d98f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:18.110727 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abc2a99-dc09-444b-bfd7-c302177d98f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:18.112082 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abc2a99-dc09-444b-bfd7-c302177d98f7 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:18.196071 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:18.206773 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 633/1282] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Sep 26 10:23:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 10:23:18.212810 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239c673-b328-4099-b37b-894d18aa310f None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:18.270809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:18.307568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239c673-b328-4099-b37b-894d18aa310f None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:18.340567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d651e-ad7d-4bb0-b05a-0202b8ee40a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:18.341175 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d651e-ad7d-4bb0-b05a-0202b8ee40a7 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:18.341455 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d651e-ad7d-4bb0-b05a-0202b8ee40a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:18.341724 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d651e-ad7d-4bb0-b05a-0202b8ee40a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:18.351268 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d651e-ad7d-4bb0-b05a-0202b8ee40a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:18.352685 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d651e-ad7d-4bb0-b05a-0202b8ee40a7 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:18.366678 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-652d651e-ad7d-4bb0-b05a-0202b8ee40a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:23:18.366678 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:23:18.366678 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:23:18.366678 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:23:18.366678 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:23:18.366678 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:23:18.366678 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:23:18.366678 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:23:18.366678 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:23:18.366678 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:23:18.366678 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:23:18.366678 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:23:18.366678 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 10:23:18.366678 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 10:23:18.366678 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 10:23:18.366678 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 10:23:18.367788 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:23:18.367788 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:23:18.367788 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:23:18.367788 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:23:18.367788 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:23:18.367788 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:23:18.367788 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:23:18.367788 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:23:18.367788 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:23:18.367788 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:23:18.367788 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:23:18.367788 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:23:18.367788 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:23:18.367788 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:23:18.367788 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:23:18.367788 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:23:18.367788 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 10:23:18.368712 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 10:23:18.368712 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 10:23:18.368712 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:23:18.368712 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 650/1283] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:23:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:23:18.373785 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc2a99-dc09-444b-bfd7-c302177d98f7 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:18.470817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc2a99-dc09-444b-bfd7-c302177d98f7 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:18.503693 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b845a551-b85f-4309-b0b4-45ddf23ca921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:18.504290 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b845a551-b85f-4309-b0b4-45ddf23ca921 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:18.504569 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b845a551-b85f-4309-b0b4-45ddf23ca921 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:18.504845 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b845a551-b85f-4309-b0b4-45ddf23ca921 None admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:18.505477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b845a551-b85f-4309-b0b4-45ddf23ca921 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:23:18.505777 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b845a551-b85f-4309-b0b4-45ddf23ca921 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:18.507160 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b845a551-b85f-4309-b0b4-45ddf23ca921 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:18.518565 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 634/1284] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Sep 26 10:23:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:23:18.524154 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d651e-ad7d-4bb0-b05a-0202b8ee40a7 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:18.619346 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d651e-ad7d-4bb0-b05a-0202b8ee40a7 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:18.632627 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:18.652624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c260615-d5df-40cc-92b6-b8de805a983d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:18.653226 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c260615-d5df-40cc-92b6-b8de805a983d None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:18.653503 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c260615-d5df-40cc-92b6-b8de805a983d None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:18.653771 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c260615-d5df-40cc-92b6-b8de805a983d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:18.660176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c260615-d5df-40cc-92b6-b8de805a983d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:18.661587 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c260615-d5df-40cc-92b6-b8de805a983d None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:18.672413 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application [None req-1c260615-d5df-40cc-92b6-b8de805a983d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:23:18.672413 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:23:18.672413 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:23:18.672413 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:23:18.672413 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:23:18.672413 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:23:18.672413 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:23:18.672413 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:23:18.672413 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:23:18.672413 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:23:18.672413 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:23:18.672413 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:23:18.672413 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 10:23:18.672413 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 10:23:18.672413 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 10:23:18.673453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 10:23:18.673453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:23:18.673453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:23:18.673453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:23:18.673453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:23:18.673453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:23:18.673453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:23:18.673453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:23:18.673453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application with Lock( Sep 26 10:23:18.673453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:23:18.673453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:23:18.673453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:23:18.673453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:23:18.673453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:23:18.673453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:23:18.673453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:23:18.673453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:23:18.674441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 10:23:18.674441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 10:23:18.674441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 10:23:18.674441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 10:23:18.674441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 10:23:18.674441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 10:23:18.674441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 10:23:18.674441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: ERROR keystone.server.flask.application Sep 26 10:23:18.674441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 651/1285] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Sep 26 10:23:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:23:18.679175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b845a551-b85f-4309-b0b4-45ddf23ca921 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:18.728458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:18.777837 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b845a551-b85f-4309-b0b4-45ddf23ca921 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:18.811798 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97081b-072e-4990-9836-ea1459dc00ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:18.812396 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97081b-072e-4990-9836-ea1459dc00ae None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:18.812697 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97081b-072e-4990-9836-ea1459dc00ae None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:18.812968 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97081b-072e-4990-9836-ea1459dc00ae None admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:18.813650 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97081b-072e-4990-9836-ea1459dc00ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:23:18.813951 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97081b-072e-4990-9836-ea1459dc00ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:18.815399 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97081b-072e-4990-9836-ea1459dc00ae None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:18.827865 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 635/1286] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Sep 26 10:23:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:23:18.833833 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c260615-d5df-40cc-92b6-b8de805a983d None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:18.930630 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c260615-d5df-40cc-92b6-b8de805a983d None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:18.963477 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70295c5-4bbf-4f81-97fb-708a6d8dedd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:18.964089 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70295c5-4bbf-4f81-97fb-708a6d8dedd2 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:18.964368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70295c5-4bbf-4f81-97fb-708a6d8dedd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:18.964638 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70295c5-4bbf-4f81-97fb-708a6d8dedd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:18.965376 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70295c5-4bbf-4f81-97fb-708a6d8dedd2 None admin] RBAC: Adding query filter params (role.id=532b37d9076c45618cf352aafb85f35e, scope.project.id=cff3a6011f4c4893bddffdad3d69f3df, user.id=84b02e7351914cabb67462454b328d05) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:23:18.965679 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70295c5-4bbf-4f81-97fb-708a6d8dedd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:18.967118 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70295c5-4bbf-4f81-97fb-708a6d8dedd2 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:18.972660 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 652/1287] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Sep 26 10:23:18 2022] GET /identity/v3/role_assignments?user.id=84b02e7351914cabb67462454b328d05&scope.project.id=cff3a6011f4c4893bddffdad3d69f3df&role.id=532b37d9076c45618cf352aafb85f35e => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:23:21.154817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492b037-d765-4cd8-942c-4f12590476e2 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:21.155332 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492b037-d765-4cd8-942c-4f12590476e2 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:21.155332 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492b037-d765-4cd8-942c-4f12590476e2 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:21.156701 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 636/1288] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:23:21.161898 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afeb7ac-963e-4e93-9571-64bbce3ad888 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:21.162438 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afeb7ac-963e-4e93-9571-64bbce3ad888 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:21.162531 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afeb7ac-963e-4e93-9571-64bbce3ad888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:21.180105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-0afeb7ac-963e-4e93-9571-64bbce3ad888 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:23:21.181268 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-0afeb7ac-963e-4e93-9571-64bbce3ad888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:23:21.209639 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:21.245560 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 653/1289] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:23:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:23:21.521582 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492b037-d765-4cd8-942c-4f12590476e2 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:21.619720 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492b037-d765-4cd8-942c-4f12590476e2 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:21.630514 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:21.654434 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c7f2e-dc4e-4720-b3bc-7b4beee7623e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:21.655047 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c7f2e-dc4e-4720-b3bc-7b4beee7623e None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:21.655325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c7f2e-dc4e-4720-b3bc-7b4beee7623e None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:21.655593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c7f2e-dc4e-4720-b3bc-7b4beee7623e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:21.656162 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c7f2e-dc4e-4720-b3bc-7b4beee7623e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:21.657515 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c7f2e-dc4e-4720-b3bc-7b4beee7623e None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:21.666154 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-5b0c7f2e-dc4e-4720-b3bc-7b4beee7623e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 10:23:21.666154 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:23:21.666154 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 10:23:21.666154 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 10:23:21.666154 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 10:23:21.666154 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 10:23:21.666154 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 10:23:21.666154 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:23:21.666154 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 10:23:21.666154 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:23:21.666154 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 10:23:21.666154 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:23:21.666154 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:23:21.666154 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:23:21.666154 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:23:21.666154 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:23:21.666154 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:23:21.667395 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:23:21.667395 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:23:21.667395 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:23:21.667395 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:23:21.667395 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:23:21.667395 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:23:21.667395 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 10:23:21.667395 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 10:23:21.667395 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 10:23:21.667395 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 10:23:21.667395 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:23:21.667395 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:23:21.667395 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:23:21.667395 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:23:21.667395 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:23:21.667395 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:23:21.668494 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:23:21.668494 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:23:21.668494 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:23:21.668494 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:23:21.668494 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:23:21.668494 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:23:21.668494 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:23:21.668494 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:23:21.668494 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:23:21.668494 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:23:21.668494 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 10:23:21.668494 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:23:21.668494 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 10:23:21.668494 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:23:21.668494 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 637/1290] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Sep 26 10:23:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:23:21.673157 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afeb7ac-963e-4e93-9571-64bbce3ad888 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:21.768422 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afeb7ac-963e-4e93-9571-64bbce3ad888 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:21.801818 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769e8d0-a7d3-4a25-8c48-567e4623649c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:21.802458 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769e8d0-a7d3-4a25-8c48-567e4623649c None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:21.802742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769e8d0-a7d3-4a25-8c48-567e4623649c None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:21.803016 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769e8d0-a7d3-4a25-8c48-567e4623649c None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:21.803645 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769e8d0-a7d3-4a25-8c48-567e4623649c None admin] RBAC: Adding query filter params (name=placement) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:23:21.804420 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769e8d0-a7d3-4a25-8c48-567e4623649c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:21.805290 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769e8d0-a7d3-4a25-8c48-567e4623649c None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:21.810687 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 654/1291] 10.222.0.49 () {56 vars in 1116 bytes} [Mon Sep 26 10:23:21 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:23:21.816269 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c7f2e-dc4e-4720-b3bc-7b4beee7623e None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:21.912466 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c7f2e-dc4e-4720-b3bc-7b4beee7623e None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:21.945218 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8c0e2-3a6c-42d6-a1d0-58ef3ca84cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:21.945801 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8c0e2-3a6c-42d6-a1d0-58ef3ca84cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:21.946104 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8c0e2-3a6c-42d6-a1d0-58ef3ca84cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:21.946377 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8c0e2-3a6c-42d6-a1d0-58ef3ca84cb5 None admin] PATH_INFO: `/v3/services` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:21.946995 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8c0e2-3a6c-42d6-a1d0-58ef3ca84cb5 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:23:21.947295 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8c0e2-3a6c-42d6-a1d0-58ef3ca84cb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:21.948633 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8c0e2-3a6c-42d6-a1d0-58ef3ca84cb5 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:21.952863 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 638/1292] 10.222.0.49 () {56 vars in 1116 bytes} [Mon Sep 26 10:23:21 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:23:24.068025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc24ae-9e2e-4c85-b4a4-a3e4ff929cb9 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:24.068519 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc24ae-9e2e-4c85-b4a4-a3e4ff929cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:24.068519 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc24ae-9e2e-4c85-b4a4-a3e4ff929cb9 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:24.069812 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 655/1293] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:23:24.075052 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db3d1a-b30e-4bf9-9393-e14a6af1a00d None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:24.075443 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db3d1a-b30e-4bf9-9393-e14a6af1a00d None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:24.075601 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db3d1a-b30e-4bf9-9393-e14a6af1a00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:24.094643 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-c5db3d1a-b30e-4bf9-9393-e14a6af1a00d None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:23:24.096097 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-c5db3d1a-b30e-4bf9-9393-e14a6af1a00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:23:24.126082 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:24.162368 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 639/1294] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:23:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:23:24.434382 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc24ae-9e2e-4c85-b4a4-a3e4ff929cb9 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:24.465399 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:24.533526 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc24ae-9e2e-4c85-b4a4-a3e4ff929cb9 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:24.566626 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd70ac56-6136-4f76-8d85-44d4d9831aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:24.567218 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70ac56-6136-4f76-8d85-44d4d9831aab None admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:24.567495 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70ac56-6136-4f76-8d85-44d4d9831aab None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:24.567766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70ac56-6136-4f76-8d85-44d4d9831aab None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:24.568405 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd70ac56-6136-4f76-8d85-44d4d9831aab None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:24.569747 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd70ac56-6136-4f76-8d85-44d4d9831aab None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:24.583070 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 656/1295] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Sep 26 10:23:24 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:23:26.718220 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de549c71-0a56-41b1-b548-5f9672b64e3e None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:26.718807 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de549c71-0a56-41b1-b548-5f9672b64e3e None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:26.718807 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de549c71-0a56-41b1-b548-5f9672b64e3e None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:26.720129 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 640/1296] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:23:26.725671 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59159551-46d8-445a-aaad-afdcfa68bc4f None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:26.726118 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59159551-46d8-445a-aaad-afdcfa68bc4f None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:26.726246 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59159551-46d8-445a-aaad-afdcfa68bc4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:26.744068 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-59159551-46d8-445a-aaad-afdcfa68bc4f None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:23:26.745246 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-59159551-46d8-445a-aaad-afdcfa68bc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:23:26.816625 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 657/1297] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:23:26 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:23:27.083255 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de549c71-0a56-41b1-b548-5f9672b64e3e None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:27.181911 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de549c71-0a56-41b1-b548-5f9672b64e3e None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:27.214819 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b30214-c379-49c6-8b81-b3fadcac091b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:27.215424 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b30214-c379-49c6-8b81-b3fadcac091b None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:27.215695 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b30214-c379-49c6-8b81-b3fadcac091b None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:27.215962 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b30214-c379-49c6-8b81-b3fadcac091b None admin] PATH_INFO: `/v3/services/placement` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:27.216607 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b30214-c379-49c6-8b81-b3fadcac091b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:27.217957 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b30214-c379-49c6-8b81-b3fadcac091b None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:27.226482 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-e8b30214-c379-49c6-8b81-b3fadcac091b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 10:23:27.226482 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:23:27.226482 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 10:23:27.226482 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 10:23:27.226482 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 10:23:27.226482 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 10:23:27.226482 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 10:23:27.226482 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:23:27.226482 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 10:23:27.226482 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:23:27.226482 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 10:23:27.226482 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:23:27.226482 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:23:27.226482 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:23:27.226482 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:23:27.226482 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:23:27.226482 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:23:27.227663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:23:27.227663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:23:27.227663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:23:27.227663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:23:27.227663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:23:27.227663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:23:27.227663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 10:23:27.227663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 10:23:27.227663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 10:23:27.227663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 10:23:27.227663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:23:27.227663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:23:27.227663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:23:27.227663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:23:27.227663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:23:27.227663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:23:27.228621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:23:27.228621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:23:27.228621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:23:27.228621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:23:27.228621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:23:27.228621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:23:27.228621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:23:27.228621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:23:27.228621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:23:27.228621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:23:27.228621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 10:23:27.228621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:23:27.228621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 10:23:27.228621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:23:27.228621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 641/1298] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Sep 26 10:23:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:23:27.233923 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59159551-46d8-445a-aaad-afdcfa68bc4f None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:27.252076 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:27.331535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59159551-46d8-445a-aaad-afdcfa68bc4f None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:27.364519 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d7069-866d-47f9-82d1-1e16177b2979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:27.365063 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d7069-866d-47f9-82d1-1e16177b2979 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:27.365341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d7069-866d-47f9-82d1-1e16177b2979 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:27.365612 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d7069-866d-47f9-82d1-1e16177b2979 None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:27.366263 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6d7069-866d-47f9-82d1-1e16177b2979 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:23:27.366568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6d7069-866d-47f9-82d1-1e16177b2979 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:27.367920 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6d7069-866d-47f9-82d1-1e16177b2979 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:27.373452 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 658/1299] 10.222.0.49 () {56 vars in 1116 bytes} [Mon Sep 26 10:23:27 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:23:27.379284 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b30214-c379-49c6-8b81-b3fadcac091b None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:27.410265 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:27.483276 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b30214-c379-49c6-8b81-b3fadcac091b None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:27.522293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff543d5-13e2-4d36-8239-68f76ab60b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:27.522894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff543d5-13e2-4d36-8239-68f76ab60b9a None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:27.523175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff543d5-13e2-4d36-8239-68f76ab60b9a None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:27.523444 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff543d5-13e2-4d36-8239-68f76ab60b9a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:27.524139 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff543d5-13e2-4d36-8239-68f76ab60b9a None admin] RBAC: Adding query filter params (interface=public, service_id=ff28e1e9da414b6db40f96ce5e615295, region_id=RegionOne) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:23:27.524443 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff543d5-13e2-4d36-8239-68f76ab60b9a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:27.525808 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff543d5-13e2-4d36-8239-68f76ab60b9a None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:27.531173 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 642/1300] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Sep 26 10:23:27 2022] GET /identity/v3/endpoints?service_id=ff28e1e9da414b6db40f96ce5e615295&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:23:27.536973 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d7069-866d-47f9-82d1-1e16177b2979 None admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:27.616910 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:27.634526 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d7069-866d-47f9-82d1-1e16177b2979 None admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:27.666250 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4d2f7-b441-4c9d-950b-c1d537b0e196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:27.666837 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4d2f7-b441-4c9d-950b-c1d537b0e196 None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:27.667110 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4d2f7-b441-4c9d-950b-c1d537b0e196 None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:27.667379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4d2f7-b441-4c9d-950b-c1d537b0e196 None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:27.668555 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d4d2f7-b441-4c9d-950b-c1d537b0e196 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:27.669896 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d4d2f7-b441-4c9d-950b-c1d537b0e196 None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:27.676451 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 659/1301] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Sep 26 10:23:27 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:23:29.795076 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de383e58-d468-4724-84a1-bb5e3af451b1 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:29.795600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de383e58-d468-4724-84a1-bb5e3af451b1 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:29.795600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de383e58-d468-4724-84a1-bb5e3af451b1 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:29.796929 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 643/1302] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:23:29.802316 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af8412-81f9-4ff0-89c8-61ac3c46330b None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:29.802712 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af8412-81f9-4ff0-89c8-61ac3c46330b None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:29.802869 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af8412-81f9-4ff0-89c8-61ac3c46330b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:29.820692 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-91af8412-81f9-4ff0-89c8-61ac3c46330b None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:23:29.821879 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-91af8412-81f9-4ff0-89c8-61ac3c46330b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:23:29.886796 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 660/1303] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:23:29 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 10:23:30.152879 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de383e58-d468-4724-84a1-bb5e3af451b1 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:30.244558 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:30.251632 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de383e58-d468-4724-84a1-bb5e3af451b1 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:30.284172 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcadf3ed-a371-4bfa-9f0f-6e163672d717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:30.284778 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcadf3ed-a371-4bfa-9f0f-6e163672d717 None admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:30.285050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcadf3ed-a371-4bfa-9f0f-6e163672d717 None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:30.285317 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcadf3ed-a371-4bfa-9f0f-6e163672d717 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:30.285881 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcadf3ed-a371-4bfa-9f0f-6e163672d717 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:30.287251 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcadf3ed-a371-4bfa-9f0f-6e163672d717 None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:30.295706 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application [None req-fcadf3ed-a371-4bfa-9f0f-6e163672d717 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 10:23:30.295706 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:23:30.295706 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 10:23:30.295706 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 10:23:30.295706 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 10:23:30.295706 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 10:23:30.295706 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 10:23:30.295706 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:23:30.295706 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 10:23:30.295706 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:23:30.295706 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 10:23:30.295706 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:23:30.295706 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 10:23:30.295706 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 10:23:30.295706 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 10:23:30.295706 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 10:23:30.295706 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 10:23:30.296919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 10:23:30.296919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 10:23:30.296919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 10:23:30.296919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 10:23:30.296919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 10:23:30.296919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 10:23:30.296919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 10:23:30.296919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 10:23:30.296919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 10:23:30.296919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 10:23:30.296919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 10:23:30.296919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 10:23:30.296919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 10:23:30.296919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 10:23:30.296919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 10:23:30.296919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 10:23:30.297788 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 10:23:30.297788 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application with Lock( Sep 26 10:23:30.297788 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 10:23:30.297788 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self._enter() Sep 26 10:23:30.297788 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 10:23:30.297788 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 10:23:30.297788 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 10:23:30.297788 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application return self.creator() Sep 26 10:23:30.297788 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 10:23:30.297788 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application created_value = creator( Sep 26 10:23:30.297788 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 10:23:30.297788 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 10:23:30.297788 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 10:23:30.297788 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: ERROR keystone.server.flask.application Sep 26 10:23:30.297788 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 644/1304] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Sep 26 10:23:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 10:23:30.302637 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af8412-81f9-4ff0-89c8-61ac3c46330b None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:30.377359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:30.400215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af8412-81f9-4ff0-89c8-61ac3c46330b None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:30.435033 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02c672-5844-4d77-8ea3-73ca47c5613b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:30.435611 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02c672-5844-4d77-8ea3-73ca47c5613b None admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:30.435897 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02c672-5844-4d77-8ea3-73ca47c5613b None admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:30.436167 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02c672-5844-4d77-8ea3-73ca47c5613b None admin] PATH_INFO: `/v3/services` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:30.436800 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02c672-5844-4d77-8ea3-73ca47c5613b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:23:30.437207 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02c672-5844-4d77-8ea3-73ca47c5613b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:30.438517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02c672-5844-4d77-8ea3-73ca47c5613b None admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:30.443679 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 661/1305] 10.222.0.49 () {56 vars in 1116 bytes} [Mon Sep 26 10:23:30 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:23:30.450680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcadf3ed-a371-4bfa-9f0f-6e163672d717 None admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:23:30.546570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcadf3ed-a371-4bfa-9f0f-6e163672d717 None admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:23:30.579707 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9f14d-b791-447d-9048-951a5889d4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:23:30.580287 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9f14d-b791-447d-9048-951a5889d4bb None admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:23:30.580560 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9f14d-b791-447d-9048-951a5889d4bb None admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:23:30.580827 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9f14d-b791-447d-9048-951a5889d4bb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:23:30.581482 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9f14d-b791-447d-9048-951a5889d4bb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:23:30.582846 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9f14d-b791-447d-9048-951a5889d4bb None admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:23:30.601550 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 645/1306] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:23:30 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:23:40.440605 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=30 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:23:40.597400 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:24:34.543893 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac12b07-7c5e-4c15-abd7-7eb9db9956c6 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:24:34.544452 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac12b07-7c5e-4c15-abd7-7eb9db9956c6 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:24:34.544452 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac12b07-7c5e-4c15-abd7-7eb9db9956c6 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:24:34.552163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d384a-13f6-4f63-8908-d956c2080728 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:24:34.553047 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 662/1307] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:24:34 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:24:34.553287 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d384a-13f6-4f63-8908-d956c2080728 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:24:34.553770 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d384a-13f6-4f63-8908-d956c2080728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:24:34.631371 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-745d384a-13f6-4f63-8908-d956c2080728 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:24:34.632586 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-745d384a-13f6-4f63-8908-d956c2080728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:24:34.738135 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 646/1308] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:24:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:24:35.033317 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf7605-f476-4f51-92fd-b31562aa9c70 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:24:35.033679 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf7605-f476-4f51-92fd-b31562aa9c70 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:24:35.034230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf7605-f476-4f51-92fd-b31562aa9c70 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:24:35.035652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 663/1309] 10.222.0.49 () {54 vars in 964 bytes} [Mon Sep 26 10:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:24:35.041667 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb7eef-a2cf-4c4b-80ec-430abab1022b None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:24:35.042054 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb7eef-a2cf-4c4b-80ec-430abab1022b None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:24:35.042430 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb7eef-a2cf-4c4b-80ec-430abab1022b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:24:35.122766 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-f5bb7eef-a2cf-4c4b-80ec-430abab1022b None None] MFA Rules not processed for user `f841bf31c43b4333a3d834d38b5197e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:24:35.123990 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-f5bb7eef-a2cf-4c4b-80ec-430abab1022b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:24:35.149169 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:24:35.215696 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 647/1310] 10.222.0.49 () {58 vars in 1061 bytes} [Mon Sep 26 10:24:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:24:35.225093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf7605-f476-4f51-92fd-b31562aa9c70 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:24:35.226962 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-ecdf7605-f476-4f51-92fd-b31562aa9c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:24:35.355200 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf7605-f476-4f51-92fd-b31562aa9c70 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:24:35.454410 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1739fd1c-228f-4202-9a26-1a21e789e9a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:24:35.455045 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739fd1c-228f-4202-9a26-1a21e789e9a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:24:35.455325 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739fd1c-228f-4202-9a26-1a21e789e9a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:24:35.455597 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739fd1c-228f-4202-9a26-1a21e789e9a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:24:35.457287 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-1739fd1c-228f-4202-9a26-1a21e789e9a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:24:35.518793 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:24:35.578461 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1739fd1c-228f-4202-9a26-1a21e789e9a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:24:35.579794 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1739fd1c-228f-4202-9a26-1a21e789e9a1 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:24:35.600528 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 664/1311] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:24:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:24:38.031321 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d991c-280f-4f60-843b-4c87966ce2fd None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:24:38.031852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d991c-280f-4f60-843b-4c87966ce2fd None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:24:38.031852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d991c-280f-4f60-843b-4c87966ce2fd None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:24:38.033183 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 648/1312] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:24:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:24:38.038317 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708c96a-436a-40a1-a6d5-381fca600b0f None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:24:38.038739 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708c96a-436a-40a1-a6d5-381fca600b0f None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:24:38.038865 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708c96a-436a-40a1-a6d5-381fca600b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:24:38.121102 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-6708c96a-436a-40a1-a6d5-381fca600b0f None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:24:38.122267 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-6708c96a-436a-40a1-a6d5-381fca600b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:24:38.217364 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 665/1313] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:24:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:24:38.507291 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d991c-280f-4f60-843b-4c87966ce2fd None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:24:38.508818 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-500d991c-280f-4f60-843b-4c87966ce2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:24:38.615366 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d991c-280f-4f60-843b-4c87966ce2fd None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:24:38.687094 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:24:38.689519 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe6524-d180-4cbb-ac71-c871579ecbe6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:24:38.690361 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe6524-d180-4cbb-ac71-c871579ecbe6 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:24:38.690810 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe6524-d180-4cbb-ac71-c871579ecbe6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:24:38.691263 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe6524-d180-4cbb-ac71-c871579ecbe6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:24:38.692877 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-17fe6524-d180-4cbb-ac71-c871579ecbe6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:24:38.804811 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe6524-d180-4cbb-ac71-c871579ecbe6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:24:38.806280 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe6524-d180-4cbb-ac71-c871579ecbe6 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:24:38.825817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 649/1314] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:24:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:24:41.356222 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c57dd9-b2ad-43fb-b352-47c16b48f886 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:24:41.356800 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c57dd9-b2ad-43fb-b352-47c16b48f886 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:24:41.356800 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c57dd9-b2ad-43fb-b352-47c16b48f886 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:24:41.358137 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 666/1315] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:24:41.363281 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a531a-8f78-4015-94ba-1d0115caa36a None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:24:41.363723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a531a-8f78-4015-94ba-1d0115caa36a None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:24:41.363828 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a531a-8f78-4015-94ba-1d0115caa36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:24:41.428968 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-f37a531a-8f78-4015-94ba-1d0115caa36a None None] MFA Rules not processed for user `4cf56ce5f5694a0499199ccd4ee6ab75`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:24:41.430203 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-f37a531a-8f78-4015-94ba-1d0115caa36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:24:41.481608 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:24:41.525083 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 650/1316] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:24:41 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:24:41.809704 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c57dd9-b2ad-43fb-b352-47c16b48f886 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:24:41.811272 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-63c57dd9-b2ad-43fb-b352-47c16b48f886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:24:41.821888 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:24:41.914456 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c57dd9-b2ad-43fb-b352-47c16b48f886 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:24:41.985624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22e577-6b1c-406b-a7d0-8c45e5bd9549 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:24:41.986249 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22e577-6b1c-406b-a7d0-8c45e5bd9549 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:24:41.986526 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22e577-6b1c-406b-a7d0-8c45e5bd9549 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:24:41.986798 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22e577-6b1c-406b-a7d0-8c45e5bd9549 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:24:41.988376 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-7d22e577-6b1c-406b-a7d0-8c45e5bd9549 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:24:42.048837 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:24:42.094640 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d22e577-6b1c-406b-a7d0-8c45e5bd9549 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:24:42.095988 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d22e577-6b1c-406b-a7d0-8c45e5bd9549 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:24:42.115168 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 667/1317] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:24:41 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:24:44.776214 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2f0e4-664c-4a34-a3f7-1f07c8cb44a4 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:24:44.777293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2f0e4-664c-4a34-a3f7-1f07c8cb44a4 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:24:44.777293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2f0e4-664c-4a34-a3f7-1f07c8cb44a4 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:24:44.778670 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 651/1318] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:24:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:24:44.783738 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495e01c-eaba-486d-be02-5c88b1d4cf55 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:24:44.784127 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495e01c-eaba-486d-be02-5c88b1d4cf55 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:24:44.784285 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495e01c-eaba-486d-be02-5c88b1d4cf55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:24:44.848784 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-d495e01c-eaba-486d-be02-5c88b1d4cf55 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:24:44.849980 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-d495e01c-eaba-486d-be02-5c88b1d4cf55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:24:44.942899 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 668/1319] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:24:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:24:45.222906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2f0e4-664c-4a34-a3f7-1f07c8cb44a4 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:24:45.224442 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-f1a2f0e4-664c-4a34-a3f7-1f07c8cb44a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:24:45.326544 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2f0e4-664c-4a34-a3f7-1f07c8cb44a4 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:24:45.398520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f1e33-507c-448c-95ef-e96099d17048 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:24:45.399115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f1e33-507c-448c-95ef-e96099d17048 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:24:45.399392 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f1e33-507c-448c-95ef-e96099d17048 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:24:45.399661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f1e33-507c-448c-95ef-e96099d17048 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:24:45.401247 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-df1f1e33-507c-448c-95ef-e96099d17048 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:24:45.408018 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:24:45.511767 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f1e33-507c-448c-95ef-e96099d17048 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:24:45.513197 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f1e33-507c-448c-95ef-e96099d17048 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:24:45.530699 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 652/1320] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:24:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:24:48.285271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c40a16-5e1c-45ef-8b6b-258e1d5b2bef None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:24:48.285843 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c40a16-5e1c-45ef-8b6b-258e1d5b2bef None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:24:48.285843 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c40a16-5e1c-45ef-8b6b-258e1d5b2bef None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:24:48.287232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 669/1321] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:24:48.293039 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21dd17f-36aa-43a6-87fa-7457941893c7 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:24:48.293441 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21dd17f-36aa-43a6-87fa-7457941893c7 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:24:48.293584 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21dd17f-36aa-43a6-87fa-7457941893c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:24:48.363884 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-a21dd17f-36aa-43a6-87fa-7457941893c7 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:24:48.365257 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-a21dd17f-36aa-43a6-87fa-7457941893c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:24:48.429585 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:24:48.474443 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 653/1322] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:24:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:24:48.754502 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c40a16-5e1c-45ef-8b6b-258e1d5b2bef None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:24:48.756045 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-f4c40a16-5e1c-45ef-8b6b-258e1d5b2bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:24:48.776409 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:24:48.865688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c40a16-5e1c-45ef-8b6b-258e1d5b2bef None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:24:48.939539 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3632727-3196-4462-8c27-e9fc5b91099e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:24:48.940142 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3632727-3196-4462-8c27-e9fc5b91099e service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:24:48.940420 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3632727-3196-4462-8c27-e9fc5b91099e service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:24:48.940690 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3632727-3196-4462-8c27-e9fc5b91099e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:24:48.942312 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-c3632727-3196-4462-8c27-e9fc5b91099e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:24:49.015677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:24:49.054223 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3632727-3196-4462-8c27-e9fc5b91099e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:24:49.055703 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3632727-3196-4462-8c27-e9fc5b91099e service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:24:49.074215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 670/1323] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:24:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:24:51.257140 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594dd0b3-5552-4cff-86a9-e512f8f7c2c1 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:24:51.257695 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594dd0b3-5552-4cff-86a9-e512f8f7c2c1 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:24:51.257695 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594dd0b3-5552-4cff-86a9-e512f8f7c2c1 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:24:51.259055 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 654/1324] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:24:51.264645 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b56873-9d58-470d-8550-b20f881a7fa8 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:24:51.265093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b56873-9d58-470d-8550-b20f881a7fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:24:51.265225 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b56873-9d58-470d-8550-b20f881a7fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:24:51.331116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-f8b56873-9d58-470d-8550-b20f881a7fa8 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:24:51.332324 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-f8b56873-9d58-470d-8550-b20f881a7fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:24:51.435447 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 671/1325] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:24:51.715169 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594dd0b3-5552-4cff-86a9-e512f8f7c2c1 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:24:51.716986 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-594dd0b3-5552-4cff-86a9-e512f8f7c2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:24:51.824252 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594dd0b3-5552-4cff-86a9-e512f8f7c2c1 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:24:51.898593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6821c-ddd6-475a-bb3a-e5b808b5f3ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:24:51.899190 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6821c-ddd6-475a-bb3a-e5b808b5f3ae service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:24:51.899464 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6821c-ddd6-475a-bb3a-e5b808b5f3ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:24:51.899732 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6821c-ddd6-475a-bb3a-e5b808b5f3ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:24:51.901335 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-70a6821c-ddd6-475a-bb3a-e5b808b5f3ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:24:51.918367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:24:52.011505 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a6821c-ddd6-475a-bb3a-e5b808b5f3ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:24:52.012939 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a6821c-ddd6-475a-bb3a-e5b808b5f3ae service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:24:52.031593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 655/1326] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:24:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:24:54.219369 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744104e-8b7d-43b6-8866-536df24a8547 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:24:54.219883 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744104e-8b7d-43b6-8866-536df24a8547 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:24:54.219975 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744104e-8b7d-43b6-8866-536df24a8547 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:24:54.221252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 672/1327] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:24:54.226373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3f2bc-34b8-4046-b8ae-3926b68e7111 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:24:54.226762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3f2bc-34b8-4046-b8ae-3926b68e7111 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:24:54.226948 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3f2bc-34b8-4046-b8ae-3926b68e7111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:24:54.293374 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-eaa3f2bc-34b8-4046-b8ae-3926b68e7111 None None] MFA Rules not processed for user `4cf56ce5f5694a0499199ccd4ee6ab75`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:24:54.294563 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-eaa3f2bc-34b8-4046-b8ae-3926b68e7111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:24:54.359528 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:24:54.391368 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 656/1328] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:24:54 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:24:54.688934 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32981470-0925-460a-b771-8f562f6a521b None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:24:54.689222 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32981470-0925-460a-b771-8f562f6a521b None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:24:54.689494 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32981470-0925-460a-b771-8f562f6a521b None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:24:54.690890 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 673/1329] 10.222.0.49 () {54 vars in 964 bytes} [Mon Sep 26 10:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:24:54.697222 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16fe72-ecad-444d-a3e5-b20059d3e933 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:24:54.697505 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16fe72-ecad-444d-a3e5-b20059d3e933 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:24:54.697778 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16fe72-ecad-444d-a3e5-b20059d3e933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:24:54.788555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-cd16fe72-ecad-444d-a3e5-b20059d3e933 None None] MFA Rules not processed for user `f841bf31c43b4333a3d834d38b5197e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:24:54.789776 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-cd16fe72-ecad-444d-a3e5-b20059d3e933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:24:54.869283 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 657/1330] 10.222.0.49 () {58 vars in 1061 bytes} [Mon Sep 26 10:24:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:24:54.878941 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32981470-0925-460a-b771-8f562f6a521b None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:24:54.880402 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-32981470-0925-460a-b771-8f562f6a521b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:24:54.911348 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:24:54.990591 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32981470-0925-460a-b771-8f562f6a521b None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:24:55.064495 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8462f28-cf10-465c-8391-00f71375fd04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:24:55.065103 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8462f28-cf10-465c-8391-00f71375fd04 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:24:55.065385 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8462f28-cf10-465c-8391-00f71375fd04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:24:55.065667 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8462f28-cf10-465c-8391-00f71375fd04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:24:55.067284 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-b8462f28-cf10-465c-8391-00f71375fd04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:24:55.143555 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:24:55.176298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8462f28-cf10-465c-8391-00f71375fd04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:24:55.177727 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8462f28-cf10-465c-8391-00f71375fd04 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:24:55.195574 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 674/1331] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:24:54 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:24:57.982545 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7a1c7-95b5-45bc-a1e0-e0bcc41c6d97 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:24:57.983072 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7a1c7-95b5-45bc-a1e0-e0bcc41c6d97 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:24:57.983072 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7a1c7-95b5-45bc-a1e0-e0bcc41c6d97 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:24:57.984429 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 658/1332] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:24:57.989647 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4364d-9b16-45ca-9fa4-116b74ef056b None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:24:57.990107 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4364d-9b16-45ca-9fa4-116b74ef056b None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:24:57.990246 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4364d-9b16-45ca-9fa4-116b74ef056b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:24:58.055557 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-0ce4364d-9b16-45ca-9fa4-116b74ef056b None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:24:58.056771 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-0ce4364d-9b16-45ca-9fa4-116b74ef056b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:24:58.151742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 675/1333] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:24:58.429902 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7a1c7-95b5-45bc-a1e0-e0bcc41c6d97 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:24:58.431477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-d4d7a1c7-95b5-45bc-a1e0-e0bcc41c6d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:24:58.504491 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:24:58.539571 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7a1c7-95b5-45bc-a1e0-e0bcc41c6d97 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:24:58.612204 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544eabf-c282-4754-82ab-2d9079fd7896 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:24:58.612795 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544eabf-c282-4754-82ab-2d9079fd7896 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:24:58.613067 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544eabf-c282-4754-82ab-2d9079fd7896 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:24:58.613331 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544eabf-c282-4754-82ab-2d9079fd7896 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:24:58.614934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-e544eabf-c282-4754-82ab-2d9079fd7896 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:24:58.725702 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e544eabf-c282-4754-82ab-2d9079fd7896 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:24:58.727093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e544eabf-c282-4754-82ab-2d9079fd7896 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:24:58.739121 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:24:58.748293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 659/1334] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:24:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:25:00.975055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043fb29c-d448-49e6-bd2b-f578e3751904 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:00.975616 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043fb29c-d448-49e6-bd2b-f578e3751904 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:00.975616 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043fb29c-d448-49e6-bd2b-f578e3751904 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:00.977004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 676/1335] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:25:00.982876 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583509e5-d87c-467f-b264-982106d5aeb5 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:00.983320 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583509e5-d87c-467f-b264-982106d5aeb5 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:00.983424 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583509e5-d87c-467f-b264-982106d5aeb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:01.047859 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-583509e5-d87c-467f-b264-982106d5aeb5 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:25:01.049052 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-583509e5-d87c-467f-b264-982106d5aeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:01.144506 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 660/1336] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:25:01.436193 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043fb29c-d448-49e6-bd2b-f578e3751904 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:25:01.437734 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-043fb29c-d448-49e6-bd2b-f578e3751904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:01.479348 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:01.543481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043fb29c-d448-49e6-bd2b-f578e3751904 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:25:01.616456 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acbec3-1cfc-4d44-b4ac-2e075da57649 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:25:01.617055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acbec3-1cfc-4d44-b4ac-2e075da57649 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:01.617334 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acbec3-1cfc-4d44-b4ac-2e075da57649 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:01.617606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acbec3-1cfc-4d44-b4ac-2e075da57649 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:01.619251 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-10acbec3-1cfc-4d44-b4ac-2e075da57649 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:01.701603 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:01.727237 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10acbec3-1cfc-4d44-b4ac-2e075da57649 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:25:01.728664 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10acbec3-1cfc-4d44-b4ac-2e075da57649 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:25:01.747521 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 677/1337] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:25:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:25:04.357785 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e4d86-323d-424e-9658-5e552cff9dd6 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:04.358381 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e4d86-323d-424e-9658-5e552cff9dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:04.358381 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e4d86-323d-424e-9658-5e552cff9dd6 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:04.359685 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 661/1338] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:25:04.364732 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98b9eb-4070-450f-b6bc-dce8b290e284 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:04.365181 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98b9eb-4070-450f-b6bc-dce8b290e284 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:04.365286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98b9eb-4070-450f-b6bc-dce8b290e284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:04.430328 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-bf98b9eb-4070-450f-b6bc-dce8b290e284 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:25:04.431566 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-bf98b9eb-4070-450f-b6bc-dce8b290e284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:04.535358 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 678/1339] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:25:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:25:04.821886 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e4d86-323d-424e-9658-5e552cff9dd6 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:25:04.824012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-635e4d86-323d-424e-9658-5e552cff9dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:04.898972 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:04.926038 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e4d86-323d-424e-9658-5e552cff9dd6 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:25:04.998543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a021ef6-ebbe-46f4-8758-a7b1120e3338 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:25:04.999140 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a021ef6-ebbe-46f4-8758-a7b1120e3338 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:04.999414 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a021ef6-ebbe-46f4-8758-a7b1120e3338 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:04.999677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a021ef6-ebbe-46f4-8758-a7b1120e3338 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:05.001240 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-4a021ef6-ebbe-46f4-8758-a7b1120e3338 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:05.105631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a021ef6-ebbe-46f4-8758-a7b1120e3338 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:25:05.107001 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a021ef6-ebbe-46f4-8758-a7b1120e3338 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:25:05.124678 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 662/1340] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:25:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:25:09.278629 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcbf71-b8a7-49c0-872b-61721b087ad5 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:09.279195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcbf71-b8a7-49c0-872b-61721b087ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:09.279195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcbf71-b8a7-49c0-872b-61721b087ad5 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:09.280673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 679/1341] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:25:09.285785 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c9cb1-3ebf-441b-a4f7-4be6e18342d2 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:09.286249 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c9cb1-3ebf-441b-a4f7-4be6e18342d2 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:09.286424 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c9cb1-3ebf-441b-a4f7-4be6e18342d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:09.291878 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:09.367666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-c52c9cb1-3ebf-441b-a4f7-4be6e18342d2 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:25:09.368869 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-c52c9cb1-3ebf-441b-a4f7-4be6e18342d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:09.475638 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 663/1342] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:25:09.760207 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfcbf71-b8a7-49c0-872b-61721b087ad5 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:25:09.762294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-2bfcbf71-b8a7-49c0-872b-61721b087ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:09.815465 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:09.872879 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfcbf71-b8a7-49c0-872b-61721b087ad5 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:25:09.946003 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da794e-836a-48ce-bc32-353bfacc52ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:25:09.946625 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da794e-836a-48ce-bc32-353bfacc52ab service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:09.946901 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da794e-836a-48ce-bc32-353bfacc52ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:09.947172 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da794e-836a-48ce-bc32-353bfacc52ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:09.949289 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-50da794e-836a-48ce-bc32-353bfacc52ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:10.042450 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:10.058982 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da794e-836a-48ce-bc32-353bfacc52ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:25:10.060410 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da794e-836a-48ce-bc32-353bfacc52ab service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:25:10.079579 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 680/1343] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:25:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:25:12.843473 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d82542-ece2-4688-9e6e-0efd04d064b9 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:12.843994 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d82542-ece2-4688-9e6e-0efd04d064b9 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:12.844105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d82542-ece2-4688-9e6e-0efd04d064b9 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:12.845500 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 664/1344] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:25:12.850757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443cdbc-15a2-4dd8-8764-4c44b34500a5 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:12.851211 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443cdbc-15a2-4dd8-8764-4c44b34500a5 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:12.851309 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443cdbc-15a2-4dd8-8764-4c44b34500a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:12.919583 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-1443cdbc-15a2-4dd8-8764-4c44b34500a5 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:25:12.920800 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-1443cdbc-15a2-4dd8-8764-4c44b34500a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:13.017688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 681/1345] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:25:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:25:13.305555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d82542-ece2-4688-9e6e-0efd04d064b9 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:25:13.307121 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-42d82542-ece2-4688-9e6e-0efd04d064b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:13.410553 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:13.429485 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d82542-ece2-4688-9e6e-0efd04d064b9 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:25:13.523654 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d825a-8899-4437-acac-4f0ce2e3fa26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:25:13.524266 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d825a-8899-4437-acac-4f0ce2e3fa26 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:13.524547 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d825a-8899-4437-acac-4f0ce2e3fa26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:13.524816 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d825a-8899-4437-acac-4f0ce2e3fa26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:13.527095 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-850d825a-8899-4437-acac-4f0ce2e3fa26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:13.637278 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d825a-8899-4437-acac-4f0ce2e3fa26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:25:13.638679 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d825a-8899-4437-acac-4f0ce2e3fa26 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:25:13.658068 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 665/1346] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:25:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:25:18.054503 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d73f5-74b8-42db-94f1-c9d976d2e52f None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:18.054984 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d73f5-74b8-42db-94f1-c9d976d2e52f None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:18.055066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d73f5-74b8-42db-94f1-c9d976d2e52f None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:18.058677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 682/1347] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:25:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:25:18.061860 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a52aa9-680b-48c6-8768-3fb09fa6048a None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:18.062339 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a52aa9-680b-48c6-8768-3fb09fa6048a None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:18.062485 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a52aa9-680b-48c6-8768-3fb09fa6048a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:18.083384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:18.147578 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-85a52aa9-680b-48c6-8768-3fb09fa6048a None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:25:18.148820 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-85a52aa9-680b-48c6-8768-3fb09fa6048a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:18.263372 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 666/1348] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:25:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:25:18.544586 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d73f5-74b8-42db-94f1-c9d976d2e52f None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:25:18.546149 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-637d73f5-74b8-42db-94f1-c9d976d2e52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:18.603114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:18.649014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d73f5-74b8-42db-94f1-c9d976d2e52f None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:25:18.721331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5496e01-6c36-4cd6-8872-7c1da79430de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:25:18.722300 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5496e01-6c36-4cd6-8872-7c1da79430de service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:18.722576 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5496e01-6c36-4cd6-8872-7c1da79430de service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:18.722843 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5496e01-6c36-4cd6-8872-7c1da79430de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:18.724404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-e5496e01-6c36-4cd6-8872-7c1da79430de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:18.830253 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:18.833862 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5496e01-6c36-4cd6-8872-7c1da79430de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:25:18.835902 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5496e01-6c36-4cd6-8872-7c1da79430de service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:25:18.854945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 683/1349] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:25:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:25:23.753272 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649f222-6ae2-4127-a44d-73a8726e2a88 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:23.753750 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649f222-6ae2-4127-a44d-73a8726e2a88 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:23.753879 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649f222-6ae2-4127-a44d-73a8726e2a88 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:23.755559 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 667/1350] 10.222.0.49 () {54 vars in 921 bytes} [Mon Sep 26 10:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:25:23.760693 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51feb8-f375-4223-b2a4-43d551605abc None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:23.761128 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51feb8-f375-4223-b2a4-43d551605abc None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:23.763549 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51feb8-f375-4223-b2a4-43d551605abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:23.876565 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-ea51feb8-f375-4223-b2a4-43d551605abc None None] MFA Rules not processed for user `84b02e7351914cabb67462454b328d05`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:25:23.885128 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-ea51feb8-f375-4223-b2a4-43d551605abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:24.043357 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 684/1351] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Sep 26 10:25:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:25:28.211369 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d51337-053b-46b0-a513-f2b47b8a32de None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:28.212193 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d51337-053b-46b0-a513-f2b47b8a32de None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:28.212478 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d51337-053b-46b0-a513-f2b47b8a32de None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:28.214411 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 668/1352] 10.222.0.49 () {54 vars in 921 bytes} [Mon Sep 26 10:25:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:25:28.221321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a530c3-2412-41ae-8c1b-fce4de5ced5a None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:28.221789 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a530c3-2412-41ae-8c1b-fce4de5ced5a None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:28.222371 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a530c3-2412-41ae-8c1b-fce4de5ced5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:28.262557 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=59 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:28.281142 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:28.306508 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-c5a530c3-2412-41ae-8c1b-fce4de5ced5a None None] MFA Rules not processed for user `84b02e7351914cabb67462454b328d05`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:25:28.308402 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-c5a530c3-2412-41ae-8c1b-fce4de5ced5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:28.419575 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 685/1353] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Sep 26 10:25:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:25:29.753439 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ff0b8-43f3-4e52-994b-4c35a631b33a None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:29.753965 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ff0b8-43f3-4e52-994b-4c35a631b33a None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:29.753965 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ff0b8-43f3-4e52-994b-4c35a631b33a None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:29.755329 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 669/1354] 10.222.0.49 () {54 vars in 921 bytes} [Mon Sep 26 10:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:25:29.760397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01e679-731e-4bbc-bf15-44f909199b4e None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:29.760796 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01e679-731e-4bbc-bf15-44f909199b4e None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:29.761065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01e679-731e-4bbc-bf15-44f909199b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:29.846492 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-9e01e679-731e-4bbc-bf15-44f909199b4e None None] MFA Rules not processed for user `84b02e7351914cabb67462454b328d05`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:25:29.847506 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-9e01e679-731e-4bbc-bf15-44f909199b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:29.890052 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:29.955756 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 686/1355] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Sep 26 10:25:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:25:33.574182 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7128b77-a0d7-44d0-85ef-651c001fbb57 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:33.574691 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7128b77-a0d7-44d0-85ef-651c001fbb57 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:33.574691 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7128b77-a0d7-44d0-85ef-651c001fbb57 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:33.575953 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 670/1356] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:25:33.581352 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27f4e9-071e-457b-9ad0-77e45f81fbbe None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:33.581791 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27f4e9-071e-457b-9ad0-77e45f81fbbe None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:33.581885 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27f4e9-071e-457b-9ad0-77e45f81fbbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:33.655308 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-9d27f4e9-071e-457b-9ad0-77e45f81fbbe None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:25:33.656511 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-9d27f4e9-071e-457b-9ad0-77e45f81fbbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:33.766165 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 687/1357] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:25:34.071779 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514de885-94b6-4152-86e3-2b2fc239c254 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:34.072028 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514de885-94b6-4152-86e3-2b2fc239c254 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:34.072299 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514de885-94b6-4152-86e3-2b2fc239c254 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:34.073895 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 671/1358] 10.222.0.49 () {54 vars in 961 bytes} [Mon Sep 26 10:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:25:34.083388 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcda875-3a62-499a-9dbd-327aafad7106 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:34.084033 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcda875-3a62-499a-9dbd-327aafad7106 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:34.084518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcda875-3a62-499a-9dbd-327aafad7106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:34.107734 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:34.172751 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-3bcda875-3a62-499a-9dbd-327aafad7106 None None] MFA Rules not processed for user `cee41014d5d043888c6b399710bccab4`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:25:34.173851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-3bcda875-3a62-499a-9dbd-327aafad7106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:34.309335 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 688/1359] 10.222.0.49 () {58 vars in 1058 bytes} [Mon Sep 26 10:25:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:25:34.317150 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514de885-94b6-4152-86e3-2b2fc239c254 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:25:34.318719 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-514de885-94b6-4152-86e3-2b2fc239c254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:34.475545 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514de885-94b6-4152-86e3-2b2fc239c254 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:25:34.568005 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:34.614585 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bcb74-5850-4f8f-ba5a-20f83dd3ed0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:25:34.615270 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bcb74-5850-4f8f-ba5a-20f83dd3ed0d service nova] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:34.615500 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bcb74-5850-4f8f-ba5a-20f83dd3ed0d service nova] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:34.615775 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bcb74-5850-4f8f-ba5a-20f83dd3ed0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:34.617362 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-c58bcb74-5850-4f8f-ba5a-20f83dd3ed0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:34.717244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58bcb74-5850-4f8f-ba5a-20f83dd3ed0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:25:34.718867 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58bcb74-5850-4f8f-ba5a-20f83dd3ed0d service nova] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:25:34.742489 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 672/1360] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:25:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:25:37.114656 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e3147-135e-4714-9641-d9f83dd0b6e8 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:37.115258 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e3147-135e-4714-9641-d9f83dd0b6e8 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:37.115863 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e3147-135e-4714-9641-d9f83dd0b6e8 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:37.126126 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 689/1361] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:25:37 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:25:37.128888 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d5d54-3045-4e14-b73d-50d666ccf0d9 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:37.129340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d5d54-3045-4e14-b73d-50d666ccf0d9 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:37.129340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d5d54-3045-4e14-b73d-50d666ccf0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:37.197455 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-490d5d54-3045-4e14-b73d-50d666ccf0d9 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:25:37.198724 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-490d5d54-3045-4e14-b73d-50d666ccf0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:37.239663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:37.302899 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 673/1362] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:25:37.593309 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e3147-135e-4714-9641-d9f83dd0b6e8 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:25:37.594866 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-2c7e3147-135e-4714-9641-d9f83dd0b6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:37.668185 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:37.775638 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e3147-135e-4714-9641-d9f83dd0b6e8 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:25:37.916045 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc993ad-8d28-4b15-8f11-5801bfc7414e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:25:37.917517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc993ad-8d28-4b15-8f11-5801bfc7414e service nova] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:37.917791 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc993ad-8d28-4b15-8f11-5801bfc7414e service nova] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:37.918087 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc993ad-8d28-4b15-8f11-5801bfc7414e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:37.919716 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-2cc993ad-8d28-4b15-8f11-5801bfc7414e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:37.943630 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:38.015905 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc993ad-8d28-4b15-8f11-5801bfc7414e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:25:38.017538 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc993ad-8d28-4b15-8f11-5801bfc7414e service nova] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:25:38.035775 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 690/1363] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:25:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:25:40.537611 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f35fba-a535-45dc-b159-8a45b701432e None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:40.538231 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f35fba-a535-45dc-b159-8a45b701432e None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:40.538231 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f35fba-a535-45dc-b159-8a45b701432e None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:40.539655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 674/1364] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:25:40.545036 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8eb9e-1eba-425d-9eb2-1882eea0be1c None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:40.545510 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8eb9e-1eba-425d-9eb2-1882eea0be1c None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:40.545791 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8eb9e-1eba-425d-9eb2-1882eea0be1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:40.614696 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-0bd8eb9e-1eba-425d-9eb2-1882eea0be1c None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:25:40.615889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-0bd8eb9e-1eba-425d-9eb2-1882eea0be1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:40.714134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:40.722423 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 691/1365] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:25:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:25:41.007345 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e3f02-8de4-4000-b2d3-2123112cbfd1 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:41.007879 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e3f02-8de4-4000-b2d3-2123112cbfd1 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:41.008344 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e3f02-8de4-4000-b2d3-2123112cbfd1 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:41.009855 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 675/1366] 10.222.0.49 () {54 vars in 961 bytes} [Mon Sep 26 10:25:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:25:41.015655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d0b84-1616-4900-8b06-f0736c869460 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:41.016067 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d0b84-1616-4900-8b06-f0736c869460 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:41.016437 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d0b84-1616-4900-8b06-f0736c869460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:41.097528 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-7f1d0b84-1616-4900-8b06-f0736c869460 None None] MFA Rules not processed for user `cee41014d5d043888c6b399710bccab4`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:25:41.099097 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-7f1d0b84-1616-4900-8b06-f0736c869460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:41.221342 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 692/1367] 10.222.0.49 () {58 vars in 1058 bytes} [Mon Sep 26 10:25:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:25:41.228902 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e3f02-8de4-4000-b2d3-2123112cbfd1 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:25:41.230995 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-0e7e3f02-8de4-4000-b2d3-2123112cbfd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:41.384365 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e3f02-8de4-4000-b2d3-2123112cbfd1 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:25:41.406478 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:41.513801 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80713f2b-4bca-4945-99e1-d8520c67fa06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:25:41.514441 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80713f2b-4bca-4945-99e1-d8520c67fa06 service nova] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:41.514720 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80713f2b-4bca-4945-99e1-d8520c67fa06 service nova] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:41.514987 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80713f2b-4bca-4945-99e1-d8520c67fa06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:41.516605 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-80713f2b-4bca-4945-99e1-d8520c67fa06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:41.612212 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80713f2b-4bca-4945-99e1-d8520c67fa06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:25:41.613811 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80713f2b-4bca-4945-99e1-d8520c67fa06 service nova] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:25:41.633446 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 676/1368] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:25:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:25:41.822502 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4971d4-4b43-4798-af20-2be73fa43923 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:41.823682 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4971d4-4b43-4798-af20-2be73fa43923 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:41.823682 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4971d4-4b43-4798-af20-2be73fa43923 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:41.829505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 693/1369] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:25:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:25:41.831275 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b914005-bfc1-4702-ae79-9a3ce4881ebd None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:41.831585 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b914005-bfc1-4702-ae79-9a3ce4881ebd None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:41.831839 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b914005-bfc1-4702-ae79-9a3ce4881ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:41.866186 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:41.908811 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-9b914005-bfc1-4702-ae79-9a3ce4881ebd None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:25:41.910197 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-9b914005-bfc1-4702-ae79-9a3ce4881ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:42.023215 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 677/1370] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:25:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:25:42.371849 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797808a8-b03c-4169-bf19-41c7c4d09edd None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:42.371849 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797808a8-b03c-4169-bf19-41c7c4d09edd None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:42.371849 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797808a8-b03c-4169-bf19-41c7c4d09edd None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:42.374050 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 694/1371] 10.222.0.49 () {54 vars in 963 bytes} [Mon Sep 26 10:25:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:25:42.377571 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf2e52-7fcc-4490-80e8-8a6b35366f7a None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:42.377851 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf2e52-7fcc-4490-80e8-8a6b35366f7a None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:42.378147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf2e52-7fcc-4490-80e8-8a6b35366f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:42.469067 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-07bf2e52-7fcc-4490-80e8-8a6b35366f7a None None] MFA Rules not processed for user `feddae5ba018426697854931a55104e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:25:42.470405 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-07bf2e52-7fcc-4490-80e8-8a6b35366f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:42.526441 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:42.561419 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 678/1372] 10.222.0.49 () {58 vars in 1060 bytes} [Mon Sep 26 10:25:42 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:25:42.568921 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797808a8-b03c-4169-bf19-41c7c4d09edd None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:25:42.570497 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-797808a8-b03c-4169-bf19-41c7c4d09edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:42.628310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:42.688278 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797808a8-b03c-4169-bf19-41c7c4d09edd None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:25:42.772884 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6fd7f3-495e-4fc8-832a-2365ba9f2ec5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:25:42.773760 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fd7f3-495e-4fc8-832a-2365ba9f2ec5 service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:42.773973 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fd7f3-495e-4fc8-832a-2365ba9f2ec5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:42.774347 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fd7f3-495e-4fc8-832a-2365ba9f2ec5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:42.776587 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-2b6fd7f3-495e-4fc8-832a-2365ba9f2ec5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:42.897328 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:42.914234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6fd7f3-495e-4fc8-832a-2365ba9f2ec5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:25:42.915670 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6fd7f3-495e-4fc8-832a-2365ba9f2ec5 service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:25:42.942583 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 695/1373] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:25:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:25:43.961489 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4ce5d-faac-4f36-a085-ed5d9f3aa6a6 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:43.962106 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4ce5d-faac-4f36-a085-ed5d9f3aa6a6 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:43.962106 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4ce5d-faac-4f36-a085-ed5d9f3aa6a6 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:43.963490 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 679/1374] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:25:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:25:43.969224 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606cb40-a65d-4f62-ad40-5e1fe8ef9781 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:43.969639 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606cb40-a65d-4f62-ad40-5e1fe8ef9781 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:43.969781 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606cb40-a65d-4f62-ad40-5e1fe8ef9781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:44.037222 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-9606cb40-a65d-4f62-ad40-5e1fe8ef9781 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:25:44.038486 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-9606cb40-a65d-4f62-ad40-5e1fe8ef9781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:44.253895 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 696/1375] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:25:44.543866 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4ce5d-faac-4f36-a085-ed5d9f3aa6a6 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:25:44.545516 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-eab4ce5d-faac-4f36-a085-ed5d9f3aa6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:44.694724 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4ce5d-faac-4f36-a085-ed5d9f3aa6a6 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:25:44.748272 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:44.814215 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeda3c3-621f-45d3-ba57-2c4dce5e2985 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:25:44.814831 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeda3c3-621f-45d3-ba57-2c4dce5e2985 service nova] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:44.815123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeda3c3-621f-45d3-ba57-2c4dce5e2985 service nova] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:44.815394 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeda3c3-621f-45d3-ba57-2c4dce5e2985 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:44.817004 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-3aeda3c3-621f-45d3-ba57-2c4dce5e2985 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:44.916289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeda3c3-621f-45d3-ba57-2c4dce5e2985 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:25:44.918536 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeda3c3-621f-45d3-ba57-2c4dce5e2985 service nova] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:25:44.940193 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 680/1376] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:25:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:25:47.144218 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8f551-5dd1-4ea1-98f3-36f2c939c144 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:47.144743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8f551-5dd1-4ea1-98f3-36f2c939c144 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:47.144743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8f551-5dd1-4ea1-98f3-36f2c939c144 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:47.148009 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 697/1377] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:25:47.151235 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cdfe6-9075-4bc9-bb8f-261d5f24a066 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:47.152064 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cdfe6-9075-4bc9-bb8f-261d5f24a066 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:47.152064 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cdfe6-9075-4bc9-bb8f-261d5f24a066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:47.232891 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-803cdfe6-9075-4bc9-bb8f-261d5f24a066 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:25:47.234366 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-803cdfe6-9075-4bc9-bb8f-261d5f24a066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:47.243799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:47.343896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 681/1378] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:25:47.643759 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8f551-5dd1-4ea1-98f3-36f2c939c144 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:25:47.645264 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-52e8f551-5dd1-4ea1-98f3-36f2c939c144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:47.708930 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:47.806905 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8f551-5dd1-4ea1-98f3-36f2c939c144 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:25:47.928754 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44836a8c-9d21-4de6-b290-6449c4655667 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:25:47.929536 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44836a8c-9d21-4de6-b290-6449c4655667 service nova] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:47.929876 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44836a8c-9d21-4de6-b290-6449c4655667 service nova] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:47.930251 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44836a8c-9d21-4de6-b290-6449c4655667 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:47.931924 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-44836a8c-9d21-4de6-b290-6449c4655667 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:47.951704 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:48.033911 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44836a8c-9d21-4de6-b290-6449c4655667 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:25:48.035617 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44836a8c-9d21-4de6-b290-6449c4655667 service nova] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:25:48.054501 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 698/1379] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:25:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:25:50.258766 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde4cea-d88d-406f-9014-3b122ccb68c6 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:50.259223 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde4cea-d88d-406f-9014-3b122ccb68c6 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:50.259317 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde4cea-d88d-406f-9014-3b122ccb68c6 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:50.261527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 682/1380] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:25:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:25:50.266494 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ba139-afd6-49d9-a0e3-d5a03679fcb1 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:50.266938 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ba139-afd6-49d9-a0e3-d5a03679fcb1 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:50.267144 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ba139-afd6-49d9-a0e3-d5a03679fcb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:50.365613 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-c93ba139-afd6-49d9-a0e3-d5a03679fcb1 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:25:50.368537 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-c93ba139-afd6-49d9-a0e3-d5a03679fcb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:50.486858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:50.493569 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 699/1381] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:25:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:25:50.792476 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde4cea-d88d-406f-9014-3b122ccb68c6 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:25:50.794012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-cfde4cea-d88d-406f-9014-3b122ccb68c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:50.933328 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:50.943970 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde4cea-d88d-406f-9014-3b122ccb68c6 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:25:51.063042 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd8e629-a991-40dd-be6e-5f28c9171ea3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:25:51.063569 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8e629-a991-40dd-be6e-5f28c9171ea3 service nova] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:51.063844 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8e629-a991-40dd-be6e-5f28c9171ea3 service nova] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:51.064113 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8e629-a991-40dd-be6e-5f28c9171ea3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:51.065714 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-6bd8e629-a991-40dd-be6e-5f28c9171ea3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:51.156916 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd8e629-a991-40dd-be6e-5f28c9171ea3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:25:51.158566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd8e629-a991-40dd-be6e-5f28c9171ea3 service nova] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:25:51.169168 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:51.178915 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 683/1382] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:25:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:25:53.368400 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe245672-3df6-4484-9adc-03c4b393cb03 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:53.369084 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe245672-3df6-4484-9adc-03c4b393cb03 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:53.369084 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe245672-3df6-4484-9adc-03c4b393cb03 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:53.370390 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 700/1383] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:25:53.375843 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b1c9c-6509-46cb-8127-3be053aba0ec None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:53.376323 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b1c9c-6509-46cb-8127-3be053aba0ec None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:53.376323 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b1c9c-6509-46cb-8127-3be053aba0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:53.442886 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-cd7b1c9c-6509-46cb-8127-3be053aba0ec None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:25:53.444095 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-cd7b1c9c-6509-46cb-8127-3be053aba0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:53.545911 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 684/1384] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:25:53.836035 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe245672-3df6-4484-9adc-03c4b393cb03 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:25:53.837536 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-fe245672-3df6-4484-9adc-03c4b393cb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:53.988565 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe245672-3df6-4484-9adc-03c4b393cb03 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:25:54.058731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:54.106782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb7f9b-30ef-468c-a233-d8799bba010c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:25:54.107386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb7f9b-30ef-468c-a233-d8799bba010c service nova] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:54.107658 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb7f9b-30ef-468c-a233-d8799bba010c service nova] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:54.107925 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb7f9b-30ef-468c-a233-d8799bba010c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:54.109532 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-98eb7f9b-30ef-468c-a233-d8799bba010c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:54.205443 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb7f9b-30ef-468c-a233-d8799bba010c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:25:54.207065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb7f9b-30ef-468c-a233-d8799bba010c service nova] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:25:54.228444 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 701/1385] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:25:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:25:56.412518 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaeadc5-ecdb-4888-80ca-0f7eee89e382 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:56.413089 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaeadc5-ecdb-4888-80ca-0f7eee89e382 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:56.413089 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaeadc5-ecdb-4888-80ca-0f7eee89e382 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:56.414436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 685/1386] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:25:56.419747 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738b0ef-9281-4155-a677-d75f2c281456 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:56.420193 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738b0ef-9281-4155-a677-d75f2c281456 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:56.420301 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738b0ef-9281-4155-a677-d75f2c281456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:56.488718 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-c738b0ef-9281-4155-a677-d75f2c281456 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:25:56.489924 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-c738b0ef-9281-4155-a677-d75f2c281456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:56.526850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:56.591677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 702/1387] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:25:56.883321 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaeadc5-ecdb-4888-80ca-0f7eee89e382 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:25:56.884837 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-adaeadc5-ecdb-4888-80ca-0f7eee89e382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:56.961138 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:57.031201 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaeadc5-ecdb-4888-80ca-0f7eee89e382 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:25:57.143295 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d81e12-f87e-4351-91c2-363f9e3765f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:25:57.143813 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d81e12-f87e-4351-91c2-363f9e3765f3 service nova] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:57.144089 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d81e12-f87e-4351-91c2-363f9e3765f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:57.144357 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d81e12-f87e-4351-91c2-363f9e3765f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:57.145937 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-f4d81e12-f87e-4351-91c2-363f9e3765f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:57.185129 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:57.237133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d81e12-f87e-4351-91c2-363f9e3765f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:25:57.238746 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d81e12-f87e-4351-91c2-363f9e3765f3 service nova] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:25:57.257179 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 686/1388] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:25:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:25:58.162305 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e3a1c-0d49-4e1f-ab92-9887786aa21c None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:58.162864 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e3a1c-0d49-4e1f-ab92-9887786aa21c None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:58.162864 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e3a1c-0d49-4e1f-ab92-9887786aa21c None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:58.164160 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 703/1389] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:25:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:25:58.170073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9f719-a0f3-4514-9504-963e48b65d0a None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:58.170612 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9f719-a0f3-4514-9504-963e48b65d0a None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:58.170709 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9f719-a0f3-4514-9504-963e48b65d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:58.236471 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-c6a9f719-a0f3-4514-9504-963e48b65d0a None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:25:58.237678 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-c6a9f719-a0f3-4514-9504-963e48b65d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:58.335593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 687/1390] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:25:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:25:58.635560 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21cc0a-5000-46f7-b3a0-49d13483523e None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:58.635914 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21cc0a-5000-46f7-b3a0-49d13483523e None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:58.636255 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21cc0a-5000-46f7-b3a0-49d13483523e None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:58.637671 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 704/1391] 10.222.0.49 () {54 vars in 962 bytes} [Mon Sep 26 10:25:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:25:58.642894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8540a-72f6-479b-91c4-1b77dcd36239 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:58.643182 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8540a-72f6-479b-91c4-1b77dcd36239 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:58.643460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8540a-72f6-479b-91c4-1b77dcd36239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:58.659042 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:58.721559 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-b1c8540a-72f6-479b-91c4-1b77dcd36239 None None] MFA Rules not processed for user `41b242fbe2504f8cac26ce4ff46f5390`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:25:58.722794 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-b1c8540a-72f6-479b-91c4-1b77dcd36239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:58.806165 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 688/1392] 10.222.0.49 () {58 vars in 1059 bytes} [Mon Sep 26 10:25:58 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:25:58.819803 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e21cc0a-5000-46f7-b3a0-49d13483523e None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:25:58.821398 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-1e21cc0a-5000-46f7-b3a0-49d13483523e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:58.925740 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e21cc0a-5000-46f7-b3a0-49d13483523e None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:25:58.987768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:59.003903 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0feb0-4765-4d8c-85c8-2e1505cd6859 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41b242fbe2504f8cac26ce4ff46f5390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:25:59.004512 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0feb0-4765-4d8c-85c8-2e1505cd6859 service glance] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:59.004793 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0feb0-4765-4d8c-85c8-2e1505cd6859 service glance] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:59.005099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0feb0-4765-4d8c-85c8-2e1505cd6859 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:59.006783 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-36a0feb0-4765-4d8c-85c8-2e1505cd6859 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:59.117547 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a0feb0-4765-4d8c-85c8-2e1505cd6859 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:25:59.119024 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a0feb0-4765-4d8c-85c8-2e1505cd6859 service glance] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:25:59.138097 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 705/1393] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:25:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:25:59.434983 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085f7ce-dc86-4dcc-9875-185b9b591b34 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:59.435516 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085f7ce-dc86-4dcc-9875-185b9b591b34 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:59.435516 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085f7ce-dc86-4dcc-9875-185b9b591b34 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:59.436924 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 689/1394] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:25:59.442259 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486016a-23ec-417d-b34c-d1858767cbb9 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:59.442684 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486016a-23ec-417d-b34c-d1858767cbb9 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:59.442936 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486016a-23ec-417d-b34c-d1858767cbb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:59.522068 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-2486016a-23ec-417d-b34c-d1858767cbb9 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:25:59.522944 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-2486016a-23ec-417d-b34c-d1858767cbb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:59.575155 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:25:59.636835 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 706/1395] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:25:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:25:59.889027 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf203520-19cb-45da-993b-cd8edded26d5 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:25:59.889385 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf203520-19cb-45da-993b-cd8edded26d5 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:25:59.890074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf203520-19cb-45da-993b-cd8edded26d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:25:59.925960 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486016a-23ec-417d-b34c-d1858767cbb9 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:25:59.927179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-2486016a-23ec-417d-b34c-d1858767cbb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:25:59.958994 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-bf203520-19cb-45da-993b-cd8edded26d5 None None] MFA Rules not processed for user `9158557f1a2947e2815dc8d586d8b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:25:59.960197 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-bf203520-19cb-45da-993b-cd8edded26d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:00.016513 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:00.045496 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486016a-23ec-417d-b34c-d1858767cbb9 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:26:00.048879 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 690/1396] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Sep 26 10:25:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:26:00.098331 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d35a9-895c-4eae-9c93-dfb4dc5a8420 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:00.098616 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d35a9-895c-4eae-9c93-dfb4dc5a8420 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:00.098892 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d35a9-895c-4eae-9c93-dfb4dc5a8420 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:00.100240 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 691/1397] 10.222.0.49 () {54 vars in 943 bytes} [Mon Sep 26 10:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:26:00.104452 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:00.107431 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2b72e-6d8b-4178-805a-63501c7d284c None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:00.107708 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2b72e-6d8b-4178-805a-63501c7d284c None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:00.107976 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2b72e-6d8b-4178-805a-63501c7d284c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:00.167426 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90d31e-d2a7-44c7-9d9b-6486bf506818 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:26:00.168033 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90d31e-d2a7-44c7-9d9b-6486bf506818 service nova] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:00.168312 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90d31e-d2a7-44c7-9d9b-6486bf506818 service nova] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:00.168584 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90d31e-d2a7-44c7-9d9b-6486bf506818 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:00.170224 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-6a90d31e-d2a7-44c7-9d9b-6486bf506818 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:00.205498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-faf2b72e-6d8b-4178-805a-63501c7d284c None None] MFA Rules not processed for user `5e6be34fc93a4551a83e5b8380188a6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:26:00.206782 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-faf2b72e-6d8b-4178-805a-63501c7d284c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:00.280466 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a90d31e-d2a7-44c7-9d9b-6486bf506818 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:26:00.282140 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a90d31e-d2a7-44c7-9d9b-6486bf506818 service nova] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:26:00.301221 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 707/1398] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:25:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:26:00.315444 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 692/1399] 10.222.0.49 () {58 vars in 1040 bytes} [Mon Sep 26 10:26:00 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:26:00.326276 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90d31e-d2a7-44c7-9d9b-6486bf506818 service nova] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:26:00.327614 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-6a90d31e-d2a7-44c7-9d9b-6486bf506818 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:00.398967 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:00.429004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90d31e-d2a7-44c7-9d9b-6486bf506818 service nova] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:26:00.544336 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214195f9-c554-47a8-8c29-b3f945e549ae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e6be34fc93a4551a83e5b8380188a6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:26:00.545007 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214195f9-c554-47a8-8c29-b3f945e549ae service swift] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:00.545224 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214195f9-c554-47a8-8c29-b3f945e549ae service swift] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:00.545488 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214195f9-c554-47a8-8c29-b3f945e549ae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:00.547140 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-214195f9-c554-47a8-8c29-b3f945e549ae service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:00.633449 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214195f9-c554-47a8-8c29-b3f945e549ae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:26:00.633449 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214195f9-c554-47a8-8c29-b3f945e549ae service swift] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:26:00.642234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 708/1400] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 10:26:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 10:26:02.507433 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315765d-a427-44e1-9543-7b3ab4d62d5d None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:02.507433 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315765d-a427-44e1-9543-7b3ab4d62d5d None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:02.507433 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315765d-a427-44e1-9543-7b3ab4d62d5d None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:02.508290 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 693/1401] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:26:02.513620 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d0a60-abcc-45ba-bfca-f318c6002ea8 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:02.514098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d0a60-abcc-45ba-bfca-f318c6002ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:02.514186 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d0a60-abcc-45ba-bfca-f318c6002ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:02.531719 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:02.594185 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-e76d0a60-abcc-45ba-bfca-f318c6002ea8 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:26:02.595418 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-e76d0a60-abcc-45ba-bfca-f318c6002ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:02.718110 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 709/1402] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:26:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:26:03.004912 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d315765d-a427-44e1-9543-7b3ab4d62d5d None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:26:03.006478 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-d315765d-a427-44e1-9543-7b3ab4d62d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:03.041600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:03.169674 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d315765d-a427-44e1-9543-7b3ab4d62d5d None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:26:03.289074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:03.301164 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffffc8-a06a-40be-b1ca-833df496e4cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:26:03.301779 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffffc8-a06a-40be-b1ca-833df496e4cf service nova] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:03.303156 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffffc8-a06a-40be-b1ca-833df496e4cf service nova] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:03.303156 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffffc8-a06a-40be-b1ca-833df496e4cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:03.303953 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-2bffffc8-a06a-40be-b1ca-833df496e4cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:03.409947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bffffc8-a06a-40be-b1ca-833df496e4cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:26:03.411578 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bffffc8-a06a-40be-b1ca-833df496e4cf service nova] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:26:03.432427 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 694/1403] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:26:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:26:05.698350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7b02a-63cc-46b0-9e67-b240d8074805 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:05.698911 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7b02a-63cc-46b0-9e67-b240d8074805 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:05.698911 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7b02a-63cc-46b0-9e67-b240d8074805 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:05.700138 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 710/1404] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:26:05.705344 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a3d7b-a38c-4c33-9b93-dcc9c588160f None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:05.705805 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a3d7b-a38c-4c33-9b93-dcc9c588160f None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:05.705903 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a3d7b-a38c-4c33-9b93-dcc9c588160f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:05.783732 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-8b2a3d7b-a38c-4c33-9b93-dcc9c588160f None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:26:05.785074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-8b2a3d7b-a38c-4c33-9b93-dcc9c588160f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:05.861856 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:05.903989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 695/1405] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:26:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:26:06.195169 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7b02a-63cc-46b0-9e67-b240d8074805 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:26:06.196885 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-0df7b02a-63cc-46b0-9e67-b240d8074805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:06.314046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:06.381348 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7b02a-63cc-46b0-9e67-b240d8074805 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:26:06.508791 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4e03e-280f-41db-9ab4-e28df9dbf45f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:26:06.509415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4e03e-280f-41db-9ab4-e28df9dbf45f service nova] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:06.509703 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4e03e-280f-41db-9ab4-e28df9dbf45f service nova] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:06.509995 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4e03e-280f-41db-9ab4-e28df9dbf45f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:06.511652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-64a4e03e-280f-41db-9ab4-e28df9dbf45f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:06.577887 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:06.619975 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a4e03e-280f-41db-9ab4-e28df9dbf45f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:26:06.621824 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a4e03e-280f-41db-9ab4-e28df9dbf45f service nova] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:26:06.641057 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 711/1406] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:26:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:26:08.936857 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b8b96-a274-479c-8572-1aa20073fe90 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:08.937431 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b8b96-a274-479c-8572-1aa20073fe90 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:08.937431 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b8b96-a274-479c-8572-1aa20073fe90 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:08.938958 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 696/1407] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:26:08.944843 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5341d0-b9f6-4706-b195-6266b6d5dbc5 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:08.945312 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5341d0-b9f6-4706-b195-6266b6d5dbc5 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:08.945415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5341d0-b9f6-4706-b195-6266b6d5dbc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:09.020027 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-4e5341d0-b9f6-4706-b195-6266b6d5dbc5 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:26:09.021557 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-4e5341d0-b9f6-4706-b195-6266b6d5dbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:09.130284 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 712/1408] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:26:09.430349 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b8b96-a274-479c-8572-1aa20073fe90 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:26:09.432410 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-177b8b96-a274-479c-8572-1aa20073fe90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:09.599750 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b8b96-a274-479c-8572-1aa20073fe90 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:26:09.654766 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:09.728299 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895c407-d5fc-4069-87e0-716b5ea99c4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:26:09.729065 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895c407-d5fc-4069-87e0-716b5ea99c4d service nova] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:09.729416 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895c407-d5fc-4069-87e0-716b5ea99c4d service nova] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:09.729787 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895c407-d5fc-4069-87e0-716b5ea99c4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:09.731559 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-e895c407-d5fc-4069-87e0-716b5ea99c4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:09.840678 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e895c407-d5fc-4069-87e0-716b5ea99c4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:26:09.842734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e895c407-d5fc-4069-87e0-716b5ea99c4d service nova] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:26:09.863481 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 697/1409] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:26:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:26:19.128366 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=87 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:19.860347 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=76 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:23.010622 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea03af1-5922-4201-a31d-217b41a0149d None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:23.011194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea03af1-5922-4201-a31d-217b41a0149d None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:23.011194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea03af1-5922-4201-a31d-217b41a0149d None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:23.012525 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 713/1410] 10.222.0.49 () {54 vars in 918 bytes} [Mon Sep 26 10:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:26:23.017568 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f635e-e4fe-4469-808f-b11205b6d811 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:23.017965 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f635e-e4fe-4469-808f-b11205b6d811 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:23.018227 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f635e-e4fe-4469-808f-b11205b6d811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:23.094346 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-793f635e-e4fe-4469-808f-b11205b6d811 None None] MFA Rules not processed for user `84b02e7351914cabb67462454b328d05`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:26:23.095494 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-793f635e-e4fe-4469-808f-b11205b6d811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:23.198939 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 698/1411] 10.222.0.49 () {58 vars in 1015 bytes} [Mon Sep 26 10:26:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:26:23.213566 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314349bb-f401-4520-b515-2dad67327a4b None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:23.213851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314349bb-f401-4520-b515-2dad67327a4b None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:23.214164 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314349bb-f401-4520-b515-2dad67327a4b None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:23.215781 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 714/1412] 10.222.0.49 () {54 vars in 918 bytes} [Mon Sep 26 10:26:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:26:23.220416 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e3c90-210c-4d7c-86d7-f1dab98a1582 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:23.220698 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e3c90-210c-4d7c-86d7-f1dab98a1582 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:23.220972 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e3c90-210c-4d7c-86d7-f1dab98a1582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:23.293476 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:23.296647 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-e38e3c90-210c-4d7c-86d7-f1dab98a1582 None None] MFA Rules not processed for user `cee41014d5d043888c6b399710bccab4`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:26:23.298415 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-e38e3c90-210c-4d7c-86d7-f1dab98a1582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:23.413176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 699/1413] 10.222.0.49 () {58 vars in 1015 bytes} [Mon Sep 26 10:26:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:26:26.000651 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb9319-5bd0-498c-a041-cdba68a47ed7 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:26.001209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb9319-5bd0-498c-a041-cdba68a47ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:26.001209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb9319-5bd0-498c-a041-cdba68a47ed7 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:26.002594 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 715/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 10:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:26:26.007912 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b9185-04c9-4aa5-933b-67a600f77290 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:26.008357 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b9185-04c9-4aa5-933b-67a600f77290 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:26.008455 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b9185-04c9-4aa5-933b-67a600f77290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:26.073413 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-357b9185-04c9-4aa5-933b-67a600f77290 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:26:26.074548 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-357b9185-04c9-4aa5-933b-67a600f77290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:26.128246 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:26.170844 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 700/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 10:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:26:26.472497 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb9319-5bd0-498c-a041-cdba68a47ed7 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:26:26.474214 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-dbcb9319-5bd0-498c-a041-cdba68a47ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:26.622672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb9319-5bd0-498c-a041-cdba68a47ed7 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:26:26.695220 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:26.736643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadad2a4-372f-4010-9cc5-02b3562a607d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:26:26.737176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadad2a4-372f-4010-9cc5-02b3562a607d service nova] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:26.737483 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadad2a4-372f-4010-9cc5-02b3562a607d service nova] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:26.737750 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadad2a4-372f-4010-9cc5-02b3562a607d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:26.739360 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-cadad2a4-372f-4010-9cc5-02b3562a607d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:26.828248 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadad2a4-372f-4010-9cc5-02b3562a607d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:26:26.829831 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadad2a4-372f-4010-9cc5-02b3562a607d service nova] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:26:26.847915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 716/1416] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:26:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:26:27.979404 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df2495-4f63-40a6-97ca-3a084669673a None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:27.979875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df2495-4f63-40a6-97ca-3a084669673a None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:27.979990 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df2495-4f63-40a6-97ca-3a084669673a None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:27.981268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 701/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 10:26:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:26:27.987471 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596cf552-bfab-45ea-9db9-62de60a14c91 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:27.987894 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596cf552-bfab-45ea-9db9-62de60a14c91 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:27.988021 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596cf552-bfab-45ea-9db9-62de60a14c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:28.063679 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-596cf552-bfab-45ea-9db9-62de60a14c91 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:26:28.064832 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-596cf552-bfab-45ea-9db9-62de60a14c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:28.092673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:28.161590 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 717/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 10:26:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:26:28.310051 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df2495-4f63-40a6-97ca-3a084669673a None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:26:28.311590 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-91df2495-4f63-40a6-97ca-3a084669673a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:28.456611 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df2495-4f63-40a6-97ca-3a084669673a None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:26:28.496559 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:28.570142 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71babc9-1cb1-4dc2-9b1f-34e7b38aa675 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:26:28.570748 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71babc9-1cb1-4dc2-9b1f-34e7b38aa675 service nova] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:28.571021 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71babc9-1cb1-4dc2-9b1f-34e7b38aa675 service nova] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:28.571288 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71babc9-1cb1-4dc2-9b1f-34e7b38aa675 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:28.572913 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-b71babc9-1cb1-4dc2-9b1f-34e7b38aa675 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:28.668106 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71babc9-1cb1-4dc2-9b1f-34e7b38aa675 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:26:28.669965 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71babc9-1cb1-4dc2-9b1f-34e7b38aa675 service nova] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:26:28.688079 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 702/1419] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:26:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:26:29.824462 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36082bc-f80d-4664-9c35-a8c2b04298a8 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:29.824941 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36082bc-f80d-4664-9c35-a8c2b04298a8 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:29.825038 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36082bc-f80d-4664-9c35-a8c2b04298a8 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:29.826387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 718/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 10:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 10:26:29.832582 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac10aa4-2dfc-4ad9-a005-fbfa192f2516 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:29.833001 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac10aa4-2dfc-4ad9-a005-fbfa192f2516 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:29.833160 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac10aa4-2dfc-4ad9-a005-fbfa192f2516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:29.887755 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:29.910308 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-8ac10aa4-2dfc-4ad9-a005-fbfa192f2516 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:26:29.911527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-8ac10aa4-2dfc-4ad9-a005-fbfa192f2516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:30.006433 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 703/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 10:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:26:30.168660 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36082bc-f80d-4664-9c35-a8c2b04298a8 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:26:30.170206 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-e36082bc-f80d-4664-9c35-a8c2b04298a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:30.323184 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36082bc-f80d-4664-9c35-a8c2b04298a8 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:26:30.335019 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:30.436390 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e933c-0364-4e03-a343-3282f77279ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:26:30.436990 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e933c-0364-4e03-a343-3282f77279ea service nova] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:30.437264 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e933c-0364-4e03-a343-3282f77279ea service nova] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:30.437532 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e933c-0364-4e03-a343-3282f77279ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:30.439152 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-651e933c-0364-4e03-a343-3282f77279ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:30.535942 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651e933c-0364-4e03-a343-3282f77279ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:26:30.537521 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651e933c-0364-4e03-a343-3282f77279ea service nova] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:26:30.555225 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 719/1422] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:26:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:26:31.661034 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf931c-06ab-4e06-83db-7005488ede41 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:31.661648 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf931c-06ab-4e06-83db-7005488ede41 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:31.661648 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf931c-06ab-4e06-83db-7005488ede41 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:31.663028 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 704/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 10:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:26:31.669238 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e93cdf-4d59-4b08-9826-90eb06c42a5a None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:31.669664 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e93cdf-4d59-4b08-9826-90eb06c42a5a None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:31.669766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e93cdf-4d59-4b08-9826-90eb06c42a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:31.689390 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:31.747069 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-c8e93cdf-4d59-4b08-9826-90eb06c42a5a None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:26:31.748282 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-c8e93cdf-4d59-4b08-9826-90eb06c42a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:31.851395 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 720/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 10:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:26:32.000889 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf931c-06ab-4e06-83db-7005488ede41 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:26:32.002451 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-53bf931c-06ab-4e06-83db-7005488ede41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:32.111425 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:32.147976 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf931c-06ab-4e06-83db-7005488ede41 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:26:32.260379 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6956b20-1efb-408b-8809-3c8a478c02ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:26:32.260988 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6956b20-1efb-408b-8809-3c8a478c02ad service nova] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:32.261261 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6956b20-1efb-408b-8809-3c8a478c02ad service nova] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:32.261548 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6956b20-1efb-408b-8809-3c8a478c02ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:32.263155 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-c6956b20-1efb-408b-8809-3c8a478c02ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:32.336871 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:32.350669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6956b20-1efb-408b-8809-3c8a478c02ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:26:32.352287 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6956b20-1efb-408b-8809-3c8a478c02ad service nova] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:26:32.370885 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 705/1425] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:26:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:26:33.466975 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a9721-b9e8-40b2-be47-1bc3a2eb5dfd None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:33.467533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a9721-b9e8-40b2-be47-1bc3a2eb5dfd None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:33.467533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a9721-b9e8-40b2-be47-1bc3a2eb5dfd None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:33.468881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 721/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 10:26:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:26:33.475077 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fc85a-fa14-4849-84c5-2216ef52fa23 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:33.475486 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fc85a-fa14-4849-84c5-2216ef52fa23 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:33.475598 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fc85a-fa14-4849-84c5-2216ef52fa23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:33.552185 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-3e6fc85a-fa14-4849-84c5-2216ef52fa23 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:26:33.553310 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-3e6fc85a-fa14-4849-84c5-2216ef52fa23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:33.647903 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 706/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 10:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:26:33.803171 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a9721-b9e8-40b2-be47-1bc3a2eb5dfd None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:26:33.805398 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-b22a9721-b9e8-40b2-be47-1bc3a2eb5dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:33.891470 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:33.954237 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a9721-b9e8-40b2-be47-1bc3a2eb5dfd None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:26:34.067019 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe0b9b6-ca60-47f4-8de9-2c4a06c30967 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:26:34.067541 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0b9b6-ca60-47f4-8de9-2c4a06c30967 service nova] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:34.067813 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0b9b6-ca60-47f4-8de9-2c4a06c30967 service nova] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:34.068078 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0b9b6-ca60-47f4-8de9-2c4a06c30967 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:34.069643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-8fe0b9b6-ca60-47f4-8de9-2c4a06c30967 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:34.116596 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:34.157997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe0b9b6-ca60-47f4-8de9-2c4a06c30967 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:26:34.159592 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe0b9b6-ca60-47f4-8de9-2c4a06c30967 service nova] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:26:34.177567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 722/1428] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:26:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:26:35.230235 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9913b2-e3bf-4609-8c2b-12d693e7705f None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:35.230807 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9913b2-e3bf-4609-8c2b-12d693e7705f None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:35.230807 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9913b2-e3bf-4609-8c2b-12d693e7705f None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:35.232040 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 707/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 10:26:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:26:35.238160 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3b691-81d3-4352-a014-c4d28f90640e None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:35.238636 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3b691-81d3-4352-a014-c4d28f90640e None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:35.238748 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3b691-81d3-4352-a014-c4d28f90640e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:35.313225 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-b9e3b691-81d3-4352-a014-c4d28f90640e None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:26:35.314455 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-b9e3b691-81d3-4352-a014-c4d28f90640e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:35.418385 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 723/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 10:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:26:35.569086 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9913b2-e3bf-4609-8c2b-12d693e7705f None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:26:35.570656 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-af9913b2-e3bf-4609-8c2b-12d693e7705f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:35.620417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:35.716215 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9913b2-e3bf-4609-8c2b-12d693e7705f None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:26:35.832199 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e16b36-7da4-4cc8-a1d5-1307d429b4a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:26:35.832810 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e16b36-7da4-4cc8-a1d5-1307d429b4a7 service nova] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:35.833091 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e16b36-7da4-4cc8-a1d5-1307d429b4a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:35.833355 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e16b36-7da4-4cc8-a1d5-1307d429b4a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:35.834987 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-a3e16b36-7da4-4cc8-a1d5-1307d429b4a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:35.842953 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:35.926569 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e16b36-7da4-4cc8-a1d5-1307d429b4a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:26:35.928163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e16b36-7da4-4cc8-a1d5-1307d429b4a7 service nova] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:26:35.945651 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 708/1431] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:26:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:26:37.010839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b914f57-c112-4d59-a743-63d9450241e7 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:37.011400 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b914f57-c112-4d59-a743-63d9450241e7 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:37.011400 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b914f57-c112-4d59-a743-63d9450241e7 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:37.012733 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 724/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 10:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:26:37.018897 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf6d1e-1dd2-428a-8f5e-7fb819a343f3 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:37.019301 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf6d1e-1dd2-428a-8f5e-7fb819a343f3 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:37.019453 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf6d1e-1dd2-428a-8f5e-7fb819a343f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:37.095125 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-abaf6d1e-1dd2-428a-8f5e-7fb819a343f3 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:26:37.096275 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-abaf6d1e-1dd2-428a-8f5e-7fb819a343f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:37.163552 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:37.192807 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 709/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 10:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:26:37.342404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b914f57-c112-4d59-a743-63d9450241e7 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:26:37.343921 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-8b914f57-c112-4d59-a743-63d9450241e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:37.362036 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:37.491453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b914f57-c112-4d59-a743-63d9450241e7 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:26:37.580099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:37.606692 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa583326-542c-4185-9b14-307851f57ce6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:26:37.607310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa583326-542c-4185-9b14-307851f57ce6 service nova] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:37.607592 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa583326-542c-4185-9b14-307851f57ce6 service nova] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:37.607857 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa583326-542c-4185-9b14-307851f57ce6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:37.609476 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-aa583326-542c-4185-9b14-307851f57ce6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:37.697731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa583326-542c-4185-9b14-307851f57ce6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:26:37.699274 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa583326-542c-4185-9b14-307851f57ce6 service nova] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:26:37.717606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 725/1434] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:26:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:26:38.771073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1accf176-f495-4932-a532-543d711b4678 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:38.771615 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1accf176-f495-4932-a532-543d711b4678 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:38.771615 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1accf176-f495-4932-a532-543d711b4678 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:38.772939 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 710/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 10:26:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:26:38.779874 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b138eda-82ee-447d-a819-4c287fc4cb72 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:38.780272 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b138eda-82ee-447d-a819-4c287fc4cb72 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:38.780426 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b138eda-82ee-447d-a819-4c287fc4cb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:38.857005 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-9b138eda-82ee-447d-a819-4c287fc4cb72 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:26:38.858313 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-9b138eda-82ee-447d-a819-4c287fc4cb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:38.898671 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:38.953173 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 726/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 10:26:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:26:39.104889 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1accf176-f495-4932-a532-543d711b4678 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:26:39.106338 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-1accf176-f495-4932-a532-543d711b4678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:39.251353 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1accf176-f495-4932-a532-543d711b4678 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:26:39.304368 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:39.362721 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d48d48b-b408-4a1e-937a-eb6b0f63fb00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:26:39.363247 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48d48b-b408-4a1e-937a-eb6b0f63fb00 service nova] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:39.363522 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48d48b-b408-4a1e-937a-eb6b0f63fb00 service nova] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:39.363790 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48d48b-b408-4a1e-937a-eb6b0f63fb00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:39.365367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-0d48d48b-b408-4a1e-937a-eb6b0f63fb00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:39.453126 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d48d48b-b408-4a1e-937a-eb6b0f63fb00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:26:39.454769 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d48d48b-b408-4a1e-937a-eb6b0f63fb00 service nova] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:26:39.472857 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 711/1437] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:26:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:26:40.526748 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64c3c6-73f5-4633-aaca-ba35e456a348 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:40.527317 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64c3c6-73f5-4633-aaca-ba35e456a348 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:40.527317 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64c3c6-73f5-4633-aaca-ba35e456a348 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:40.528645 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 727/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 10:26:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:26:40.534936 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf817b-0725-471f-82bd-da8ee2051ea9 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:40.535382 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf817b-0725-471f-82bd-da8ee2051ea9 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:40.535488 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf817b-0725-471f-82bd-da8ee2051ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:40.613509 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-cacf817b-0725-471f-82bd-da8ee2051ea9 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:26:40.614761 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-cacf817b-0725-471f-82bd-da8ee2051ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:40.616765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:40.710369 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 712/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 10:26:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:26:40.859352 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64c3c6-73f5-4633-aaca-ba35e456a348 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:26:40.860783 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-1b64c3c6-73f5-4633-aaca-ba35e456a348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:41.013977 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64c3c6-73f5-4633-aaca-ba35e456a348 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:26:41.040104 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:41.125443 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ee3a20-3c5b-4eef-94f8-04dda1ada707 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:26:41.125958 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee3a20-3c5b-4eef-94f8-04dda1ada707 service nova] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:41.126254 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee3a20-3c5b-4eef-94f8-04dda1ada707 service nova] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:41.126519 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee3a20-3c5b-4eef-94f8-04dda1ada707 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:41.128097 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-a5ee3a20-3c5b-4eef-94f8-04dda1ada707 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:41.214833 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ee3a20-3c5b-4eef-94f8-04dda1ada707 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:26:41.216674 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ee3a20-3c5b-4eef-94f8-04dda1ada707 service nova] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:26:41.237870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 728/1440] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:26:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:26:43.273591 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b03cc9-d2f0-4ad9-b007-5b94ef79f052 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:43.274176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b03cc9-d2f0-4ad9-b007-5b94ef79f052 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:43.274176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b03cc9-d2f0-4ad9-b007-5b94ef79f052 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:43.276083 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 713/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 10:26:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:26:43.281343 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234a6eb-cf3c-4622-817a-0bbcfa6b41f6 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:43.281787 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234a6eb-cf3c-4622-817a-0bbcfa6b41f6 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:43.281897 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234a6eb-cf3c-4622-817a-0bbcfa6b41f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:43.323074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:43.348994 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-6234a6eb-cf3c-4622-817a-0bbcfa6b41f6 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:26:43.350241 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-6234a6eb-cf3c-4622-817a-0bbcfa6b41f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:43.446950 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 729/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 10:26:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:26:43.731537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b03cc9-d2f0-4ad9-b007-5b94ef79f052 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:26:43.733138 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-a7b03cc9-d2f0-4ad9-b007-5b94ef79f052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:43.844489 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b03cc9-d2f0-4ad9-b007-5b94ef79f052 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:26:43.881094 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:43.928802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1aa47-71b8-4afb-97b0-82c351460a0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:26:43.929411 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1aa47-71b8-4afb-97b0-82c351460a0f service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:43.929687 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1aa47-71b8-4afb-97b0-82c351460a0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:43.929955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1aa47-71b8-4afb-97b0-82c351460a0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:43.932136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-9fe1aa47-71b8-4afb-97b0-82c351460a0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:44.043114 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe1aa47-71b8-4afb-97b0-82c351460a0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:26:44.044548 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe1aa47-71b8-4afb-97b0-82c351460a0f service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:26:44.064448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 714/1443] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:26:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:26:46.371202 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff8ffe-fbde-42fa-97a7-51c34076734b None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:46.371762 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff8ffe-fbde-42fa-97a7-51c34076734b None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:46.371762 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff8ffe-fbde-42fa-97a7-51c34076734b None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:46.373158 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 730/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 10:26:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:26:46.378451 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a3c74-1797-4c88-9883-63c3894759fe None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:46.378905 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a3c74-1797-4c88-9883-63c3894759fe None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:46.379012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a3c74-1797-4c88-9883-63c3894759fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:46.443991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-b06a3c74-1797-4c88-9883-63c3894759fe None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:26:46.445193 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-b06a3c74-1797-4c88-9883-63c3894759fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:46.457774 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:46.541439 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 715/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 10:26:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:26:46.826771 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff8ffe-fbde-42fa-97a7-51c34076734b None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:26:46.828330 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-acff8ffe-fbde-42fa-97a7-51c34076734b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:46.930247 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff8ffe-fbde-42fa-97a7-51c34076734b None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:26:46.943286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:47.004497 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df43265-66e2-4b43-a86b-4dc6f7197689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:26:47.005086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df43265-66e2-4b43-a86b-4dc6f7197689 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:47.005361 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df43265-66e2-4b43-a86b-4dc6f7197689 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:47.005661 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df43265-66e2-4b43-a86b-4dc6f7197689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:47.007250 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-8df43265-66e2-4b43-a86b-4dc6f7197689 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:47.113727 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df43265-66e2-4b43-a86b-4dc6f7197689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:26:47.115113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df43265-66e2-4b43-a86b-4dc6f7197689 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:26:47.133575 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 731/1446] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:26:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:26:49.641866 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743bd78-a289-470b-9692-a04cadc334c2 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:49.642442 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743bd78-a289-470b-9692-a04cadc334c2 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:49.642442 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743bd78-a289-470b-9692-a04cadc334c2 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:49.643817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 716/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 10:26:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:26:49.649775 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aba37d-723b-4cbc-8076-189f6a0d2294 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:49.650803 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aba37d-723b-4cbc-8076-189f6a0d2294 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:49.650995 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aba37d-723b-4cbc-8076-189f6a0d2294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:49.698030 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:49.716955 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-38aba37d-723b-4cbc-8076-189f6a0d2294 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:26:49.718222 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-38aba37d-723b-4cbc-8076-189f6a0d2294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:49.816182 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 732/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 10:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:26:50.101623 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743bd78-a289-470b-9692-a04cadc334c2 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:26:50.103215 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-6743bd78-a289-470b-9692-a04cadc334c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:50.206039 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743bd78-a289-470b-9692-a04cadc334c2 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:26:50.245678 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:50.279096 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f230334b-2970-4144-be79-65cb5791d2c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:26:50.279690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230334b-2970-4144-be79-65cb5791d2c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:50.279961 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230334b-2970-4144-be79-65cb5791d2c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:50.280223 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230334b-2970-4144-be79-65cb5791d2c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:50.281797 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-f230334b-2970-4144-be79-65cb5791d2c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:50.389280 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f230334b-2970-4144-be79-65cb5791d2c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:26:50.390736 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f230334b-2970-4144-be79-65cb5791d2c8 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:26:50.408683 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 717/1449] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:26:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:26:52.647029 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a8d67-2e40-4e73-8e75-cf637dd9074b None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:52.647621 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a8d67-2e40-4e73-8e75-cf637dd9074b None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:52.647621 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a8d67-2e40-4e73-8e75-cf637dd9074b None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:52.648933 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 733/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 10:26:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:26:52.654359 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee6b17-969d-4a94-9151-d067932eb4e8 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:52.654783 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee6b17-969d-4a94-9151-d067932eb4e8 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:52.654909 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee6b17-969d-4a94-9151-d067932eb4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:52.721614 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-2bee6b17-969d-4a94-9151-d067932eb4e8 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:26:52.722858 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-2bee6b17-969d-4a94-9151-d067932eb4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:52.743612 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:52.819501 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 718/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 10:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:26:53.108382 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a8d67-2e40-4e73-8e75-cf637dd9074b None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:26:53.109910 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-8d8a8d67-2e40-4e73-8e75-cf637dd9074b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:53.210924 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a8d67-2e40-4e73-8e75-cf637dd9074b None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:26:53.232290 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:53.287898 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26166a01-7e16-4dee-8f8f-84f203669eea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:26:53.288501 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26166a01-7e16-4dee-8f8f-84f203669eea service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:53.288775 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26166a01-7e16-4dee-8f8f-84f203669eea service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:53.289042 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26166a01-7e16-4dee-8f8f-84f203669eea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:53.290657 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-26166a01-7e16-4dee-8f8f-84f203669eea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:53.397705 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26166a01-7e16-4dee-8f8f-84f203669eea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:26:53.399068 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26166a01-7e16-4dee-8f8f-84f203669eea service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:26:53.417322 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 734/1452] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:26:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:26:55.603871 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077f59b-83f8-49ee-a717-af69085de0cc None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:55.604379 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077f59b-83f8-49ee-a717-af69085de0cc None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:55.604379 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077f59b-83f8-49ee-a717-af69085de0cc None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:55.605646 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 719/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 10:26:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:26:55.611015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b871c-1901-42c3-8155-96e080a17ece None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:55.611457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b871c-1901-42c3-8155-96e080a17ece None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:55.611557 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b871c-1901-42c3-8155-96e080a17ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:55.674495 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:55.677533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-259b871c-1901-42c3-8155-96e080a17ece None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:26:55.679171 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-259b871c-1901-42c3-8155-96e080a17ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:55.776245 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 735/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 10:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:26:56.062085 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077f59b-83f8-49ee-a717-af69085de0cc None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:26:56.063620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-4077f59b-83f8-49ee-a717-af69085de0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:56.172138 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077f59b-83f8-49ee-a717-af69085de0cc None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:26:56.220747 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:56.245540 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47177c8e-e12c-447d-be7f-5d7180115bc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:26:56.246170 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47177c8e-e12c-447d-be7f-5d7180115bc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:56.246448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47177c8e-e12c-447d-be7f-5d7180115bc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:56.246719 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47177c8e-e12c-447d-be7f-5d7180115bc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:56.248310 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-47177c8e-e12c-447d-be7f-5d7180115bc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:56.360516 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47177c8e-e12c-447d-be7f-5d7180115bc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:26:56.361945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47177c8e-e12c-447d-be7f-5d7180115bc2 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:26:56.379489 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 720/1455] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:26:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:26:58.567691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719ca57-baec-4a82-851f-4f49a1db3a7a None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:58.568261 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719ca57-baec-4a82-851f-4f49a1db3a7a None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:58.568261 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719ca57-baec-4a82-851f-4f49a1db3a7a None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:58.569568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 736/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 10:26:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:26:58.575010 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12de1f9-bd0a-48a8-b8da-c8df6bf3f9ea None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:58.575419 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12de1f9-bd0a-48a8-b8da-c8df6bf3f9ea None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:58.575560 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12de1f9-bd0a-48a8-b8da-c8df6bf3f9ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:58.641589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-c12de1f9-bd0a-48a8-b8da-c8df6bf3f9ea None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:26:58.642826 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-c12de1f9-bd0a-48a8-b8da-c8df6bf3f9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:58.671248 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:58.738249 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 721/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 10:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:26:59.022073 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719ca57-baec-4a82-851f-4f49a1db3a7a None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:26:59.023640 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-b719ca57-baec-4a82-851f-4f49a1db3a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:59.125422 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719ca57-baec-4a82-851f-4f49a1db3a7a None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:26:59.154278 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:26:59.200390 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0854793-ae2b-4970-a69b-bbb642d5618f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:26:59.200997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0854793-ae2b-4970-a69b-bbb642d5618f service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:26:59.201273 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0854793-ae2b-4970-a69b-bbb642d5618f service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:26:59.201546 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0854793-ae2b-4970-a69b-bbb642d5618f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:26:59.203153 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-a0854793-ae2b-4970-a69b-bbb642d5618f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:26:59.314311 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0854793-ae2b-4970-a69b-bbb642d5618f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:26:59.316390 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0854793-ae2b-4970-a69b-bbb642d5618f service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:26:59.341163 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 737/1458] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:26:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:27:01.560108 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c7e079-7a2e-424d-ad68-e09d93092532 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:01.560645 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c7e079-7a2e-424d-ad68-e09d93092532 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:01.560645 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c7e079-7a2e-424d-ad68-e09d93092532 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:01.561974 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 722/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 10:27:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:27:01.567307 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40691f-b51c-498f-9b0d-bab48877bd1f None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:01.567753 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40691f-b51c-498f-9b0d-bab48877bd1f None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:01.567874 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40691f-b51c-498f-9b0d-bab48877bd1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:01.633597 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-3b40691f-b51c-498f-9b0d-bab48877bd1f None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:27:01.634876 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-3b40691f-b51c-498f-9b0d-bab48877bd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:01.643066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:27:01.730972 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 738/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 10:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:27:02.019265 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c7e079-7a2e-424d-ad68-e09d93092532 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:27:02.020786 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-f1c7e079-7a2e-424d-ad68-e09d93092532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:02.121923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c7e079-7a2e-424d-ad68-e09d93092532 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:27:02.176161 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:27:02.193493 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd4ded-35e9-4250-adc2-cba76d3fdb7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:27:02.194116 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd4ded-35e9-4250-adc2-cba76d3fdb7e service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:02.194395 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd4ded-35e9-4250-adc2-cba76d3fdb7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:02.194661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd4ded-35e9-4250-adc2-cba76d3fdb7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:02.196225 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-c3fd4ded-35e9-4250-adc2-cba76d3fdb7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:02.302382 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd4ded-35e9-4250-adc2-cba76d3fdb7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:27:02.303815 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd4ded-35e9-4250-adc2-cba76d3fdb7e service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:27:02.321284 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 723/1461] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:27:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:27:04.504773 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99381297-ec65-4be7-ae2b-060d7224ef7b None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:04.505314 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99381297-ec65-4be7-ae2b-060d7224ef7b None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:04.505314 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99381297-ec65-4be7-ae2b-060d7224ef7b None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:04.506713 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 739/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 10:27:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:27:04.512694 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f286c8-80fd-4d23-bfab-6552fa97e31e None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:04.513159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f286c8-80fd-4d23-bfab-6552fa97e31e None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:04.513259 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f286c8-80fd-4d23-bfab-6552fa97e31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:04.579491 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-f1f286c8-80fd-4d23-bfab-6552fa97e31e None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:27:04.580714 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-f1f286c8-80fd-4d23-bfab-6552fa97e31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:04.626748 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:27:04.685423 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 724/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 10:27:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:27:04.966766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99381297-ec65-4be7-ae2b-060d7224ef7b None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:27:04.968294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-99381297-ec65-4be7-ae2b-060d7224ef7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:05.071203 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99381297-ec65-4be7-ae2b-060d7224ef7b None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:27:05.109387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:27:05.144092 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0764a366-57e0-41c1-b3cb-73c3b44175c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:27:05.144691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764a366-57e0-41c1-b3cb-73c3b44175c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:05.144970 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764a366-57e0-41c1-b3cb-73c3b44175c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:05.145242 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764a366-57e0-41c1-b3cb-73c3b44175c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:05.146856 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-0764a366-57e0-41c1-b3cb-73c3b44175c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:05.258521 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0764a366-57e0-41c1-b3cb-73c3b44175c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:27:05.260480 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0764a366-57e0-41c1-b3cb-73c3b44175c1 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:27:05.278716 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 740/1464] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:27:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:27:07.441098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822eeee-efc0-4c8a-844e-6146687f762c None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:07.441681 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822eeee-efc0-4c8a-844e-6146687f762c None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:07.441681 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822eeee-efc0-4c8a-844e-6146687f762c None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:07.443043 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 725/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 10:27:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:27:07.448547 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b67fbd-3ea3-4532-a2fd-f426aaeb967c None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:07.448956 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b67fbd-3ea3-4532-a2fd-f426aaeb967c None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:07.449062 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b67fbd-3ea3-4532-a2fd-f426aaeb967c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:07.516880 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-d4b67fbd-3ea3-4532-a2fd-f426aaeb967c None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:27:07.518167 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-d4b67fbd-3ea3-4532-a2fd-f426aaeb967c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:07.536895 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:27:07.622695 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 741/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 10:27:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:27:07.904492 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822eeee-efc0-4c8a-844e-6146687f762c None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:27:07.906044 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-9822eeee-efc0-4c8a-844e-6146687f762c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:08.007680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822eeee-efc0-4c8a-844e-6146687f762c None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:27:08.073679 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:27:08.080586 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b0d23-1314-40be-baa8-ddda04a86275 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:27:08.081193 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b0d23-1314-40be-baa8-ddda04a86275 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:08.081476 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b0d23-1314-40be-baa8-ddda04a86275 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:08.081769 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b0d23-1314-40be-baa8-ddda04a86275 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:08.083427 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-054b0d23-1314-40be-baa8-ddda04a86275 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:08.190078 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054b0d23-1314-40be-baa8-ddda04a86275 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:27:08.191509 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054b0d23-1314-40be-baa8-ddda04a86275 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:27:08.209611 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 726/1467] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:27:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:27:10.347280 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a8dee-c206-4481-a550-8cccd5129ea9 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:10.347840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a8dee-c206-4481-a550-8cccd5129ea9 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:10.347840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a8dee-c206-4481-a550-8cccd5129ea9 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:10.349231 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 742/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 10:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:27:10.354494 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566e8e4-c381-4571-8a41-e71633595e73 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:10.354937 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566e8e4-c381-4571-8a41-e71633595e73 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:10.355043 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566e8e4-c381-4571-8a41-e71633595e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:10.420241 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-a566e8e4-c381-4571-8a41-e71633595e73 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:27:10.421465 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-a566e8e4-c381-4571-8a41-e71633595e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:10.471705 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:27:10.521569 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 727/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 10:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:27:10.801849 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a8dee-c206-4481-a550-8cccd5129ea9 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:27:10.803497 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-871a8dee-c206-4481-a550-8cccd5129ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:10.907134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a8dee-c206-4481-a550-8cccd5129ea9 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:27:10.955039 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:27:10.981739 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fcdc41-95e5-4c99-b112-e97d89cb31c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:27:10.982372 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fcdc41-95e5-4c99-b112-e97d89cb31c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:10.982653 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fcdc41-95e5-4c99-b112-e97d89cb31c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:10.982923 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fcdc41-95e5-4c99-b112-e97d89cb31c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:10.984509 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-41fcdc41-95e5-4c99-b112-e97d89cb31c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:11.092835 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fcdc41-95e5-4c99-b112-e97d89cb31c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:27:11.094215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fcdc41-95e5-4c99-b112-e97d89cb31c6 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:27:11.112332 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 743/1470] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:27:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:27:20.520052 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=22 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:27:21.110202 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=66 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:27:30.942969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698dda1c-a968-431f-aa72-e1af4f61f0d0 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:30.943534 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698dda1c-a968-431f-aa72-e1af4f61f0d0 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:30.943723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698dda1c-a968-431f-aa72-e1af4f61f0d0 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:30.945022 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 728/1471] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:27:30.950765 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94f366-0cdf-4982-bc1d-d901db728ca6 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:30.951202 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94f366-0cdf-4982-bc1d-d901db728ca6 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:30.951299 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94f366-0cdf-4982-bc1d-d901db728ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:31.017630 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-3d94f366-0cdf-4982-bc1d-d901db728ca6 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:27:31.018877 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-3d94f366-0cdf-4982-bc1d-d901db728ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:31.116013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 744/1472] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:27:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:27:31.414342 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd206c8-4c25-46ea-98b9-b640178df2fe None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:31.414625 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd206c8-4c25-46ea-98b9-b640178df2fe None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:31.414898 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd206c8-4c25-46ea-98b9-b640178df2fe None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:31.416230 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 729/1473] 10.222.0.49 () {54 vars in 962 bytes} [Mon Sep 26 10:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:27:31.421043 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414c8f9-5fcc-4f26-b292-b2ee9cf32de7 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:31.421329 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414c8f9-5fcc-4f26-b292-b2ee9cf32de7 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:31.421603 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414c8f9-5fcc-4f26-b292-b2ee9cf32de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:31.498138 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-6414c8f9-5fcc-4f26-b292-b2ee9cf32de7 None None] MFA Rules not processed for user `41b242fbe2504f8cac26ce4ff46f5390`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:27:31.499334 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-6414c8f9-5fcc-4f26-b292-b2ee9cf32de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:31.519759 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:27:31.575326 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 745/1474] 10.222.0.49 () {58 vars in 1059 bytes} [Mon Sep 26 10:27:31 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:27:31.583860 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd206c8-4c25-46ea-98b9-b640178df2fe None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:27:31.585830 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-afd206c8-4c25-46ea-98b9-b640178df2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:31.688199 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd206c8-4c25-46ea-98b9-b640178df2fe None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:27:31.763191 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a773244f-cae9-4c3e-b5b4-7efec1b0ddb3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41b242fbe2504f8cac26ce4ff46f5390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:27:31.763791 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773244f-cae9-4c3e-b5b4-7efec1b0ddb3 service glance] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:31.764063 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773244f-cae9-4c3e-b5b4-7efec1b0ddb3 service glance] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:31.764326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773244f-cae9-4c3e-b5b4-7efec1b0ddb3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:31.765904 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-a773244f-cae9-4c3e-b5b4-7efec1b0ddb3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:31.819351 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:27:31.874537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a773244f-cae9-4c3e-b5b4-7efec1b0ddb3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:27:31.875965 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a773244f-cae9-4c3e-b5b4-7efec1b0ddb3 service glance] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:27:31.893973 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 730/1475] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:27:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:27:34.148793 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0294d2d-410b-4137-bd78-4d69c7031efc None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:34.149363 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0294d2d-410b-4137-bd78-4d69c7031efc None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:34.149363 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0294d2d-410b-4137-bd78-4d69c7031efc None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:34.150720 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 746/1476] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:27:34.155899 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33090390-962d-42d4-b4d5-04cbf927f3bc None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:34.156350 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33090390-962d-42d4-b4d5-04cbf927f3bc None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:34.156460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33090390-962d-42d4-b4d5-04cbf927f3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:34.221887 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-33090390-962d-42d4-b4d5-04cbf927f3bc None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:27:34.223124 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-33090390-962d-42d4-b4d5-04cbf927f3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:34.318207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 731/1477] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:27:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:27:34.590007 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0294d2d-410b-4137-bd78-4d69c7031efc None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:27:34.591852 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-b0294d2d-410b-4137-bd78-4d69c7031efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:34.717215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0294d2d-410b-4137-bd78-4d69c7031efc None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:27:34.768994 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:27:34.813955 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6753fc2-3e9d-423c-8fe2-9f50038184e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:27:34.814614 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6753fc2-3e9d-423c-8fe2-9f50038184e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:34.814886 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6753fc2-3e9d-423c-8fe2-9f50038184e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:34.815154 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6753fc2-3e9d-423c-8fe2-9f50038184e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:34.815785 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6753fc2-3e9d-423c-8fe2-9f50038184e9 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:27:34.817484 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: WARNING py.warnings [None req-e6753fc2-3e9d-423c-8fe2-9f50038184e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 10:27:34.817484 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: warnings.warn(msg) Sep 26 10:27:34.817484 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: Sep 26 10:27:34.818639 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6753fc2-3e9d-423c-8fe2-9f50038184e9 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:27:34.837197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 747/1478] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Sep 26 10:27:34 2022] GET /identity/v3/projects => generated 2553 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:27:37.015504 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b09065-cb84-4ac6-8081-b1bd85f0612e None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:37.016110 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b09065-cb84-4ac6-8081-b1bd85f0612e None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:37.016110 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b09065-cb84-4ac6-8081-b1bd85f0612e None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:37.017393 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 732/1479] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:27:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:27:37.022985 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914da90d-afc7-4cf1-8ae4-2783f2a7095a None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:37.023450 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914da90d-afc7-4cf1-8ae4-2783f2a7095a None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:37.023612 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914da90d-afc7-4cf1-8ae4-2783f2a7095a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:37.093108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-914da90d-afc7-4cf1-8ae4-2783f2a7095a None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:27:37.094235 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-914da90d-afc7-4cf1-8ae4-2783f2a7095a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:37.188640 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 748/1480] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:27:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:27:37.480414 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b09065-cb84-4ac6-8081-b1bd85f0612e None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:27:37.481949 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-72b09065-cb84-4ac6-8081-b1bd85f0612e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:37.491479 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:27:37.626045 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b09065-cb84-4ac6-8081-b1bd85f0612e None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:27:37.704854 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:27:37.737703 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5451d75c-14ae-4fc1-87c0-aa71fdc72625 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:27:37.738270 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451d75c-14ae-4fc1-87c0-aa71fdc72625 service nova] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:37.738551 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451d75c-14ae-4fc1-87c0-aa71fdc72625 service nova] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:37.738822 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451d75c-14ae-4fc1-87c0-aa71fdc72625 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:37.740410 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-5451d75c-14ae-4fc1-87c0-aa71fdc72625 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:37.841435 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5451d75c-14ae-4fc1-87c0-aa71fdc72625 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:27:37.843054 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5451d75c-14ae-4fc1-87c0-aa71fdc72625 service nova] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:27:37.861059 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 733/1481] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:27:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:27:40.176049 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce07d4-09ef-4e95-b3a4-02c656a9a411 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:40.176588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce07d4-09ef-4e95-b3a4-02c656a9a411 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:40.176588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce07d4-09ef-4e95-b3a4-02c656a9a411 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:40.177857 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 749/1482] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:27:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:27:40.183191 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf1003-0322-47a9-9a04-e093e4213897 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:40.183637 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf1003-0322-47a9-9a04-e093e4213897 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:40.183741 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf1003-0322-47a9-9a04-e093e4213897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:40.249092 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-dacf1003-0322-47a9-9a04-e093e4213897 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:27:40.250346 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-dacf1003-0322-47a9-9a04-e093e4213897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:40.296437 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:27:40.347831 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 734/1483] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:27:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:27:40.639705 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce07d4-09ef-4e95-b3a4-02c656a9a411 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:27:40.641248 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-78ce07d4-09ef-4e95-b3a4-02c656a9a411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:40.665518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:27:40.744713 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce07d4-09ef-4e95-b3a4-02c656a9a411 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:27:40.818775 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb47f4-2635-4c9f-87e4-0441090461fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:27:40.819377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb47f4-2635-4c9f-87e4-0441090461fb service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:40.819651 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb47f4-2635-4c9f-87e4-0441090461fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:40.819924 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb47f4-2635-4c9f-87e4-0441090461fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:40.821491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-75cb47f4-2635-4c9f-87e4-0441090461fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:40.897944 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:27:40.935307 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cb47f4-2635-4c9f-87e4-0441090461fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:27:40.936771 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cb47f4-2635-4c9f-87e4-0441090461fb service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:27:40.954425 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 750/1484] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:27:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:27:43.155732 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02682e8-1d63-403b-9cef-3ed7a41ce1dc None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:43.156349 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02682e8-1d63-403b-9cef-3ed7a41ce1dc None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:43.156349 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02682e8-1d63-403b-9cef-3ed7a41ce1dc None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:43.158104 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 735/1485] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:27:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:27:43.163602 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce18980-519b-4e41-9963-610f85be546a None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:43.164013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce18980-519b-4e41-9963-610f85be546a None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:43.164157 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce18980-519b-4e41-9963-610f85be546a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:43.234602 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-7ce18980-519b-4e41-9963-610f85be546a None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:27:43.235797 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-7ce18980-519b-4e41-9963-610f85be546a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:43.334102 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 751/1486] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:27:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:27:43.618570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02682e8-1d63-403b-9cef-3ed7a41ce1dc None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:27:43.620635 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-d02682e8-1d63-403b-9cef-3ed7a41ce1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:43.847793 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02682e8-1d63-403b-9cef-3ed7a41ce1dc None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:27:43.922604 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:27:43.923989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e0943-3d7e-48db-8bc2-1c4160074191 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:27:43.924591 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e0943-3d7e-48db-8bc2-1c4160074191 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:43.924884 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e0943-3d7e-48db-8bc2-1c4160074191 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:43.925149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e0943-3d7e-48db-8bc2-1c4160074191 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:43.927364 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-f93e0943-3d7e-48db-8bc2-1c4160074191 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:44.041036 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93e0943-3d7e-48db-8bc2-1c4160074191 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:27:44.042564 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93e0943-3d7e-48db-8bc2-1c4160074191 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:27:44.060973 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 736/1487] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:27:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:27:46.335833 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287e1f0-f823-409d-bec1-764825befaa1 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:46.336379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287e1f0-f823-409d-bec1-764825befaa1 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:46.336379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287e1f0-f823-409d-bec1-764825befaa1 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:46.337778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 752/1488] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:27:46.343233 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8de0eb-5678-45e9-a2a3-0ba0f439f63a None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:46.343635 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8de0eb-5678-45e9-a2a3-0ba0f439f63a None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:46.343807 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8de0eb-5678-45e9-a2a3-0ba0f439f63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:46.408882 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-4b8de0eb-5678-45e9-a2a3-0ba0f439f63a None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:27:46.410108 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-4b8de0eb-5678-45e9-a2a3-0ba0f439f63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:46.462521 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:27:46.505539 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 737/1489] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:27:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:27:46.797750 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4287e1f0-f823-409d-bec1-764825befaa1 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:27:46.799305 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-4287e1f0-f823-409d-bec1-764825befaa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:46.835519 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:27:46.902963 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4287e1f0-f823-409d-bec1-764825befaa1 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:27:46.975714 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735f6b4-2e6e-4e1b-ae51-9f33e9350c2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:27:46.976317 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735f6b4-2e6e-4e1b-ae51-9f33e9350c2e service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:46.976591 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735f6b4-2e6e-4e1b-ae51-9f33e9350c2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:46.976860 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735f6b4-2e6e-4e1b-ae51-9f33e9350c2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:46.979038 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-6735f6b4-2e6e-4e1b-ae51-9f33e9350c2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:47.058701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:27:47.088396 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735f6b4-2e6e-4e1b-ae51-9f33e9350c2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:27:47.089885 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735f6b4-2e6e-4e1b-ae51-9f33e9350c2e service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:27:47.108664 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 753/1490] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:27:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:27:49.479203 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e6a24-e688-49f8-a003-71f4326b84ad None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:49.479753 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e6a24-e688-49f8-a003-71f4326b84ad None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:49.479852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e6a24-e688-49f8-a003-71f4326b84ad None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:49.481561 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 738/1491] 10.222.0.49 () {54 vars in 927 bytes} [Mon Sep 26 10:27:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:27:49.487086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88916d-035f-445f-8e4d-776235b40b4c None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:49.487493 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88916d-035f-445f-8e4d-776235b40b4c None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:49.487664 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88916d-035f-445f-8e4d-776235b40b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:49.553581 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-1c88916d-035f-445f-8e4d-776235b40b4c None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:27:49.555139 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-1c88916d-035f-445f-8e4d-776235b40b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:49.652114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 754/1492] 10.222.0.49 () {58 vars in 1024 bytes} [Mon Sep 26 10:27:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:27:49.949693 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e6a24-e688-49f8-a003-71f4326b84ad None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:27:49.951287 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-875e6a24-e688-49f8-a003-71f4326b84ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:50.053095 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e6a24-e688-49f8-a003-71f4326b84ad None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:27:50.130805 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2efa38-d411-4d88-988b-7fee0066d67a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:27:50.131323 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2efa38-d411-4d88-988b-7fee0066d67a service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:27:50.131603 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2efa38-d411-4d88-988b-7fee0066d67a service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:27:50.131875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2efa38-d411-4d88-988b-7fee0066d67a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:27:50.133504 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-2c2efa38-d411-4d88-988b-7fee0066d67a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:27:50.143710 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:27:50.241422 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2efa38-d411-4d88-988b-7fee0066d67a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:27:50.242865 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2efa38-d411-4d88-988b-7fee0066d67a service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:27:50.261009 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 739/1493] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:27:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:27:59.649985 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=82 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:28:00.258698 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=50 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:29:03.335555 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11697b-179d-487a-9eea-10667ac24d2d None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:29:03.336130 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11697b-179d-487a-9eea-10667ac24d2d None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:29:03.336130 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11697b-179d-487a-9eea-10667ac24d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:29:03.412122 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-5a11697b-179d-487a-9eea-10667ac24d2d None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:29:03.413321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-5a11697b-179d-487a-9eea-10667ac24d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:29:03.508929 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 755/1494] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:29:03.516845 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63790d4-c251-4bca-83dd-acf4e72319ee None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:29:03.517368 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63790d4-c251-4bca-83dd-acf4e72319ee None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:29:03.517368 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63790d4-c251-4bca-83dd-acf4e72319ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:29:03.591786 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-f63790d4-c251-4bca-83dd-acf4e72319ee None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:29:03.593175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-f63790d4-c251-4bca-83dd-acf4e72319ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:29:03.687288 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 740/1495] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:29:03.691891 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a11697b-179d-487a-9eea-10667ac24d2d None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:29:03.693104 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-5a11697b-179d-487a-9eea-10667ac24d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:29:03.705164 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a11697b-179d-487a-9eea-10667ac24d2d None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:29:03.772464 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:29:03.810538 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc75ff9e-52aa-499e-abdd-f2a972d3ed61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:29:03.811154 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75ff9e-52aa-499e-abdd-f2a972d3ed61 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:29:03.811436 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75ff9e-52aa-499e-abdd-f2a972d3ed61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:29:03.811732 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75ff9e-52aa-499e-abdd-f2a972d3ed61 admin admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:29:03.812411 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc75ff9e-52aa-499e-abdd-f2a972d3ed61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:29:03.812822 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc75ff9e-52aa-499e-abdd-f2a972d3ed61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:29:03.813877 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: WARNING py.warnings [None req-dc75ff9e-52aa-499e-abdd-f2a972d3ed61 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 26 10:29:03.813877 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: warnings.warn(msg) Sep 26 10:29:03.813877 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: Sep 26 10:29:03.814808 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc75ff9e-52aa-499e-abdd-f2a972d3ed61 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:29:03.827726 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 756/1496] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:29:03.836011 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63790d4-c251-4bca-83dd-acf4e72319ee None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:29:03.846012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63790d4-c251-4bca-83dd-acf4e72319ee None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:29:03.906825 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:29:03.939234 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6a28d-0b63-4067-9931-76ce1fdd9e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:29:03.939860 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6a28d-0b63-4067-9931-76ce1fdd9e0a admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:29:03.940134 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6a28d-0b63-4067-9931-76ce1fdd9e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:29:03.940399 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6a28d-0b63-4067-9931-76ce1fdd9e0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:29:03.941054 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d6a28d-0b63-4067-9931-76ce1fdd9e0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:29:03.942844 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: WARNING py.warnings [None req-02d6a28d-0b63-4067-9931-76ce1fdd9e0a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 10:29:03.942844 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: warnings.warn(msg) Sep 26 10:29:03.942844 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: Sep 26 10:29:03.944056 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d6a28d-0b63-4067-9931-76ce1fdd9e0a admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:29:04.000775 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 741/1497] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:29:03 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:29:04.004975 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc75ff9e-52aa-499e-abdd-f2a972d3ed61 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:29:04.006584 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-dc75ff9e-52aa-499e-abdd-f2a972d3ed61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:29:04.040303 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc75ff9e-52aa-499e-abdd-f2a972d3ed61 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:29:04.132736 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485e5a8-63a9-4822-9ad0-a5bf5e9d7a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:29:04.133273 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485e5a8-63a9-4822-9ad0-a5bf5e9d7a25 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:29:04.133550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485e5a8-63a9-4822-9ad0-a5bf5e9d7a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:29:04.133818 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485e5a8-63a9-4822-9ad0-a5bf5e9d7a25 admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:29:04.134491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485e5a8-63a9-4822-9ad0-a5bf5e9d7a25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:29:04.136043 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: WARNING py.warnings [None req-6485e5a8-63a9-4822-9ad0-a5bf5e9d7a25 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 10:29:04.136043 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: warnings.warn(msg) Sep 26 10:29:04.136043 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: Sep 26 10:29:04.137246 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485e5a8-63a9-4822-9ad0-a5bf5e9d7a25 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:29:04.178371 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 757/1498] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:29:04 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:29:04.183005 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6a28d-0b63-4067-9931-76ce1fdd9e0a admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:29:04.184492 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-02d6a28d-0b63-4067-9931-76ce1fdd9e0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:29:04.245987 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6a28d-0b63-4067-9931-76ce1fdd9e0a admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:29:04.329089 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:29:04.340177 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8011e02-7f5b-44c1-98c0-bd843f444771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:29:04.340778 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8011e02-7f5b-44c1-98c0-bd843f444771 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:29:04.341049 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8011e02-7f5b-44c1-98c0-bd843f444771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:29:04.341314 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8011e02-7f5b-44c1-98c0-bd843f444771 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:29:04.341947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8011e02-7f5b-44c1-98c0-bd843f444771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:29:04.343238 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: WARNING py.warnings [None req-f8011e02-7f5b-44c1-98c0-bd843f444771 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 26 10:29:04.343238 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: warnings.warn(msg) Sep 26 10:29:04.343238 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: Sep 26 10:29:04.344126 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8011e02-7f5b-44c1-98c0-bd843f444771 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:29:04.356113 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 742/1499] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:29:04.360088 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485e5a8-63a9-4822-9ad0-a5bf5e9d7a25 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:29:04.361589 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-6485e5a8-63a9-4822-9ad0-a5bf5e9d7a25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:29:04.373360 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485e5a8-63a9-4822-9ad0-a5bf5e9d7a25 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:29:04.404199 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:29:04.473749 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3facbd-a9cd-498d-9081-9ee1f191607d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:29:04.474377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3facbd-a9cd-498d-9081-9ee1f191607d admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:29:04.474653 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3facbd-a9cd-498d-9081-9ee1f191607d admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:29:04.474920 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3facbd-a9cd-498d-9081-9ee1f191607d admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:29:04.475578 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3facbd-a9cd-498d-9081-9ee1f191607d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:29:04.476568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: WARNING py.warnings [None req-da3facbd-a9cd-498d-9081-9ee1f191607d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 26 10:29:04.476568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: warnings.warn(msg) Sep 26 10:29:04.476568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: Sep 26 10:29:04.477434 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3facbd-a9cd-498d-9081-9ee1f191607d admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:29:04.489702 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 758/1500] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:29:04.493960 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8011e02-7f5b-44c1-98c0-bd843f444771 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:29:04.496062 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-f8011e02-7f5b-44c1-98c0-bd843f444771 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:29:04.533567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8011e02-7f5b-44c1-98c0-bd843f444771 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:29:04.626188 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0165cf8-1762-4c92-b462-54056c2e5cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:29:04.626843 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0165cf8-1762-4c92-b462-54056c2e5cd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:29:04.627119 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0165cf8-1762-4c92-b462-54056c2e5cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:29:04.627389 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0165cf8-1762-4c92-b462-54056c2e5cd1 admin admin] PATH_INFO: `/v3/projects/b3e1853104aa4c7d967938c58893609b/users/90e5ff6466dd44dda58239622b95479d/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:29:04.649904 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0165cf8-1762-4c92-b462-54056c2e5cd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3e1853104aa4c7d967938c58893609b, user_id=90e5ff6466dd44dda58239622b95479d, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:29:04.651514 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: WARNING py.warnings [None req-f0165cf8-1762-4c92-b462-54056c2e5cd1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 10:29:04.651514 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: warnings.warn(msg) Sep 26 10:29:04.651514 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: Sep 26 10:29:04.653427 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0165cf8-1762-4c92-b462-54056c2e5cd1 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:29:04.676589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 743/1501] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:29:04 2022] PUT /identity/v3/projects/b3e1853104aa4c7d967938c58893609b/users/90e5ff6466dd44dda58239622b95479d/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:29:04.688629 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be0b6d-4abe-4080-8710-6192eba5205c None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:29:04.688911 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be0b6d-4abe-4080-8710-6192eba5205c None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:29:04.689181 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be0b6d-4abe-4080-8710-6192eba5205c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:29:04.746493 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-31be0b6d-4abe-4080-8710-6192eba5205c None None] MFA Rules not processed for user `90e5ff6466dd44dda58239622b95479d`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:29:04.747701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-31be0b6d-4abe-4080-8710-6192eba5205c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:29:04.827653 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 759/1502] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:29:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:29:04.841274 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ead15-41dd-4cf1-b236-04fbec9c3575 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:29:04.841557 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ead15-41dd-4cf1-b236-04fbec9c3575 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:29:04.841829 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ead15-41dd-4cf1-b236-04fbec9c3575 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:29:04.843432 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 744/1503] 10.222.0.49 () {54 vars in 963 bytes} [Mon Sep 26 10:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:29:04.848043 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94522d6e-afa0-4267-850d-4e5de2614435 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:29:04.848328 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94522d6e-afa0-4267-850d-4e5de2614435 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:29:04.848602 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94522d6e-afa0-4267-850d-4e5de2614435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:29:04.880474 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:29:04.924685 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-94522d6e-afa0-4267-850d-4e5de2614435 None None] MFA Rules not processed for user `feddae5ba018426697854931a55104e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:29:04.925883 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-94522d6e-afa0-4267-850d-4e5de2614435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:29:05.000929 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 760/1504] 10.222.0.49 () {58 vars in 1060 bytes} [Mon Sep 26 10:29:04 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:29:05.008596 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ead15-41dd-4cf1-b236-04fbec9c3575 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:29:05.010732 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-1b3ead15-41dd-4cf1-b236-04fbec9c3575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:29:05.060072 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:29:05.113295 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ead15-41dd-4cf1-b236-04fbec9c3575 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:29:05.184630 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb69138-9d99-4169-acb6-0f2a4990243e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:29:05.185253 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb69138-9d99-4169-acb6-0f2a4990243e service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:29:05.185528 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb69138-9d99-4169-acb6-0f2a4990243e service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:29:05.185799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb69138-9d99-4169-acb6-0f2a4990243e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:29:05.187396 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-9fb69138-9d99-4169-acb6-0f2a4990243e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:29:05.285051 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb69138-9d99-4169-acb6-0f2a4990243e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:29:05.286527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb69138-9d99-4169-acb6-0f2a4990243e service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:29:05.289987 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:29:05.305359 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 745/1505] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:29:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:29:05.350192 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfae9c-40ed-4f05-a09b-8758955bbec4 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:29:05.350484 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfae9c-40ed-4f05-a09b-8758955bbec4 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:29:05.350756 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfae9c-40ed-4f05-a09b-8758955bbec4 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:29:05.352023 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 761/1506] 10.222.0.49 () {52 vars in 830 bytes} [Mon Sep 26 10:29:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:29:05.356040 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb69138-9d99-4169-acb6-0f2a4990243e service cinder] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:29:05.357370 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-9fb69138-9d99-4169-acb6-0f2a4990243e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:29:05.457303 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb69138-9d99-4169-acb6-0f2a4990243e service cinder] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:29:05.549020 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a57f4e-47a3-4d02-b8f0-a6d22c7f98cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:29:05.549644 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a57f4e-47a3-4d02-b8f0-a6d22c7f98cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:29:05.549837 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a57f4e-47a3-4d02-b8f0-a6d22c7f98cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:29:05.550134 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a57f4e-47a3-4d02-b8f0-a6d22c7f98cc admin admin] PATH_INFO: `/v3/users/90e5ff6466dd44dda58239622b95479d` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:29:05.562617 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a57f4e-47a3-4d02-b8f0-a6d22c7f98cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=90e5ff6466dd44dda58239622b95479d)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:29:05.563626 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: WARNING py.warnings [None req-e9a57f4e-47a3-4d02-b8f0-a6d22c7f98cc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 10:29:05.563626 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: warnings.warn(msg) Sep 26 10:29:05.563626 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: Sep 26 10:29:05.564825 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a57f4e-47a3-4d02-b8f0-a6d22c7f98cc admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:29:05.574824 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:29:05.630937 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e9a57f4e-47a3-4d02-b8f0-a6d22c7f98cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90e5ff6466dd44dda58239622b95479d'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:29:05.631366 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e9a57f4e-47a3-4d02-b8f0-a6d22c7f98cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90e5ff6466dd44dda58239622b95479d'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:29:05.632462 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e9a57f4e-47a3-4d02-b8f0-a6d22c7f98cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90e5ff6466dd44dda58239622b95479d'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:29:05.646784 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e9a57f4e-47a3-4d02-b8f0-a6d22c7f98cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90e5ff6466dd44dda58239622b95479d'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:29:05.668537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e9a57f4e-47a3-4d02-b8f0-a6d22c7f98cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90e5ff6466dd44dda58239622b95479d'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:29:05.682878 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 746/1507] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:29:05 2022] DELETE /identity/v3/users/90e5ff6466dd44dda58239622b95479d => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:29:05.687341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2deac-0816-4766-8431-3bccf8347e73 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:29:05.687813 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2deac-0816-4766-8431-3bccf8347e73 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:29:05.687941 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2deac-0816-4766-8431-3bccf8347e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:29:05.745904 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-d3e2deac-0816-4766-8431-3bccf8347e73 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:29:05.747127 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-d3e2deac-0816-4766-8431-3bccf8347e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:29:05.822343 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:29:05.836101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 762/1508] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:29:05.847505 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a57f4e-47a3-4d02-b8f0-a6d22c7f98cc admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:29:05.848735 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-e9a57f4e-47a3-4d02-b8f0-a6d22c7f98cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:29:05.950686 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a57f4e-47a3-4d02-b8f0-a6d22c7f98cc admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:29:06.005312 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:29:06.025227 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c833942-6eed-4099-a152-cd29955dc411 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:29:06.026440 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c833942-6eed-4099-a152-cd29955dc411 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:29:06.026722 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c833942-6eed-4099-a152-cd29955dc411 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:29:06.026989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c833942-6eed-4099-a152-cd29955dc411 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:29:06.028575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-4c833942-6eed-4099-a152-cd29955dc411 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:29:06.134802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c833942-6eed-4099-a152-cd29955dc411 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:29:06.137295 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c833942-6eed-4099-a152-cd29955dc411 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:29:06.158948 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 747/1509] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:29:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:29:06.484160 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2deac-0816-4766-8431-3bccf8347e73 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:29:06.485412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-d3e2deac-0816-4766-8431-3bccf8347e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:29:06.496816 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2deac-0816-4766-8431-3bccf8347e73 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:29:06.593716 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bbc66a-6304-409b-9711-f227956c4235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:29:06.594374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bbc66a-6304-409b-9711-f227956c4235 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:29:06.594652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bbc66a-6304-409b-9711-f227956c4235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:29:06.594921 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bbc66a-6304-409b-9711-f227956c4235 admin admin] PATH_INFO: `/v3/projects/b3e1853104aa4c7d967938c58893609b` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:29:06.605494 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bbc66a-6304-409b-9711-f227956c4235 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3e1853104aa4c7d967938c58893609b)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:29:06.606543 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: WARNING py.warnings [None req-c5bbc66a-6304-409b-9711-f227956c4235 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 10:29:06.606543 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: warnings.warn(msg) Sep 26 10:29:06.606543 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: Sep 26 10:29:06.607742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bbc66a-6304-409b-9711-f227956c4235 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:29:06.666121 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c5bbc66a-6304-409b-9711-f227956c4235 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3e1853104aa4c7d967938c58893609b', 'request_id': 'req-c5bbc66a-6304-409b-9711-f227956c4235'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:29:06.679673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c5bbc66a-6304-409b-9711-f227956c4235 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3e1853104aa4c7d967938c58893609b', 'request_id': 'req-c5bbc66a-6304-409b-9711-f227956c4235'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:29:06.688677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c5bbc66a-6304-409b-9711-f227956c4235 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3e1853104aa4c7d967938c58893609b', 'request_id': 'req-c5bbc66a-6304-409b-9711-f227956c4235'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:29:06.711739 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c5bbc66a-6304-409b-9711-f227956c4235 admin admin] The token cache is being invalidate because project b3e1853104aa4c7d967938c58893609b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:29:06.712032 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c5bbc66a-6304-409b-9711-f227956c4235 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:29:06.712542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c5bbc66a-6304-409b-9711-f227956c4235 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:29:06.714494 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 763/1510] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:29:06 2022] DELETE /identity/v3/projects/b3e1853104aa4c7d967938c58893609b => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:29:16.156785 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=62 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:29:16.709711 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:30:34.829718 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153c588-8dda-4db6-8c01-80b1c39d6c6f None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:30:34.830390 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153c588-8dda-4db6-8c01-80b1c39d6c6f None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:30:34.830390 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153c588-8dda-4db6-8c01-80b1c39d6c6f None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:30:34.831640 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 748/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 10:30:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:30:34.836648 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ddb132-670e-4384-a401-c30a64c2cfeb None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:30:34.837091 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ddb132-670e-4384-a401-c30a64c2cfeb None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:30:34.837195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ddb132-670e-4384-a401-c30a64c2cfeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:30:34.924208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-64ddb132-670e-4384-a401-c30a64c2cfeb None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:30:34.925704 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-64ddb132-670e-4384-a401-c30a64c2cfeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:30:35.043219 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 764/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 10:30:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:30:35.203968 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b153c588-8dda-4db6-8c01-80b1c39d6c6f None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:30:35.206124 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-b153c588-8dda-4db6-8c01-80b1c39d6c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:30:35.330772 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b153c588-8dda-4db6-8c01-80b1c39d6c6f None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:30:35.336534 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305401c2-fb3e-41c0-9f0c-dfe1885e3f44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:30:35.337142 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305401c2-fb3e-41c0-9f0c-dfe1885e3f44 service nova] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:30:35.337412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305401c2-fb3e-41c0-9f0c-dfe1885e3f44 service nova] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:30:35.337682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305401c2-fb3e-41c0-9f0c-dfe1885e3f44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:30:35.347018 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305401c2-fb3e-41c0-9f0c-dfe1885e3f44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:30:35.348594 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305401c2-fb3e-41c0-9f0c-dfe1885e3f44 service nova] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:30:35.358309 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 749/1513] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:30:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:30:36.469353 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36047e51-e367-47f1-98c5-12d0bfc8fcba None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:30:36.470072 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36047e51-e367-47f1-98c5-12d0bfc8fcba None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:30:36.470218 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36047e51-e367-47f1-98c5-12d0bfc8fcba None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:30:36.471730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 765/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 10:30:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:30:36.478684 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dea387-6a06-435d-8147-d6a222374a26 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:30:36.479221 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dea387-6a06-435d-8147-d6a222374a26 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:30:36.479221 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dea387-6a06-435d-8147-d6a222374a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:30:36.500143 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-e0dea387-6a06-435d-8147-d6a222374a26 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:30:36.501731 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-e0dea387-6a06-435d-8147-d6a222374a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:30:36.510112 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 750/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 10:30:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:30:36.664322 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36047e51-e367-47f1-98c5-12d0bfc8fcba None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:30:36.681077 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36047e51-e367-47f1-98c5-12d0bfc8fcba None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:30:36.687126 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5895ed17-0e6d-468d-b419-7d12f3bc8480 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:30:36.687743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895ed17-0e6d-468d-b419-7d12f3bc8480 service nova] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:30:36.688025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895ed17-0e6d-468d-b419-7d12f3bc8480 service nova] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:30:36.688313 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895ed17-0e6d-468d-b419-7d12f3bc8480 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:30:36.697660 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5895ed17-0e6d-468d-b419-7d12f3bc8480 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:30:36.700004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5895ed17-0e6d-468d-b419-7d12f3bc8480 service nova] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:30:36.710458 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 766/1516] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:30:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:30:37.207861 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee6048-637b-496d-8468-e2ace2ce50b0 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:30:37.208137 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee6048-637b-496d-8468-e2ace2ce50b0 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:30:37.208407 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee6048-637b-496d-8468-e2ace2ce50b0 None None] PATH_INFO: `/v3` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:30:37.209689 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 751/1517] 10.222.0.57 () {54 vars in 925 bytes} [Mon Sep 26 10:30:37 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:30:37.239778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ae8d5-6c06-4c0f-a81b-663dfa7e16cd None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:30:37.240057 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ae8d5-6c06-4c0f-a81b-663dfa7e16cd None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:30:37.240328 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ae8d5-6c06-4c0f-a81b-663dfa7e16cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:30:37.303997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-697ae8d5-6c06-4c0f-a81b-663dfa7e16cd None None] MFA Rules not processed for user `84b02e7351914cabb67462454b328d05`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:30:37.305202 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-697ae8d5-6c06-4c0f-a81b-663dfa7e16cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:30:37.323500 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:30:37.381724 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 767/1518] 10.222.0.57 () {58 vars in 1015 bytes} [Mon Sep 26 10:30:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:30:41.079451 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c412f-6840-4896-912b-2dd8b0f42f16 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:30:41.079959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c412f-6840-4896-912b-2dd8b0f42f16 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:30:41.080085 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c412f-6840-4896-912b-2dd8b0f42f16 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:30:41.081878 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 752/1519] 10.222.0.49 () {54 vars in 961 bytes} [Mon Sep 26 10:30:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:30:41.087314 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4475ed-549a-44e3-ad4a-343546b017aa None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:30:41.087768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4475ed-549a-44e3-ad4a-343546b017aa None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:30:41.087981 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4475ed-549a-44e3-ad4a-343546b017aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:30:41.110148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-3c4475ed-549a-44e3-ad4a-343546b017aa None None] MFA Rules not processed for user `84b02e7351914cabb67462454b328d05`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:30:41.111354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-3c4475ed-549a-44e3-ad4a-343546b017aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:30:41.120066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 768/1520] 10.222.0.49 () {58 vars in 1058 bytes} [Mon Sep 26 10:30:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:30:41.128120 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c412f-6840-4896-912b-2dd8b0f42f16 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:30:41.141960 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c412f-6840-4896-912b-2dd8b0f42f16 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:30:41.147085 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa758d-c097-4609-93e4-49108f22cbe7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84b02e7351914cabb67462454b328d05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:30:41.147671 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa758d-c097-4609-93e4-49108f22cbe7 service placement] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:30:41.147945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa758d-c097-4609-93e4-49108f22cbe7 service placement] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:30:41.148214 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa758d-c097-4609-93e4-49108f22cbe7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:30:41.156411 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fa758d-c097-4609-93e4-49108f22cbe7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:30:41.157991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fa758d-c097-4609-93e4-49108f22cbe7 service placement] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:30:41.168594 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 753/1521] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:30:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:30:48.344632 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f6959-cc7c-460f-a0c1-23f32b041bdb None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:30:48.345493 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f6959-cc7c-460f-a0c1-23f32b041bdb None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:30:48.345493 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f6959-cc7c-460f-a0c1-23f32b041bdb None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:30:48.346896 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 769/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 10:30:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:30:48.353429 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f295f0-3433-4a6b-8e66-2360035e1cfe None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:30:48.353906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f295f0-3433-4a6b-8e66-2360035e1cfe None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:30:48.354053 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f295f0-3433-4a6b-8e66-2360035e1cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:30:48.373860 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-40f295f0-3433-4a6b-8e66-2360035e1cfe None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:30:48.375109 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-40f295f0-3433-4a6b-8e66-2360035e1cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:30:48.383056 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 754/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 10:30:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:30:48.537939 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f6959-cc7c-460f-a0c1-23f32b041bdb None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:30:48.553305 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f6959-cc7c-460f-a0c1-23f32b041bdb None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:30:48.558826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38280977-7190-41ca-94eb-022c23288742 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:30:48.559423 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38280977-7190-41ca-94eb-022c23288742 service nova] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:30:48.559703 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38280977-7190-41ca-94eb-022c23288742 service nova] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:30:48.559978 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38280977-7190-41ca-94eb-022c23288742 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:30:48.568112 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38280977-7190-41ca-94eb-022c23288742 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:30:48.569691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38280977-7190-41ca-94eb-022c23288742 service nova] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:30:48.579091 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 770/1524] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:30:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:30:49.662576 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e86401-8932-4167-a9de-3b83e225e674 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:30:49.663155 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e86401-8932-4167-a9de-3b83e225e674 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:30:49.663155 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e86401-8932-4167-a9de-3b83e225e674 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:30:49.664593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 755/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 10:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:30:49.671529 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea9ac7-c8d5-4a62-b7d9-dc51773e84d3 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:30:49.671966 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea9ac7-c8d5-4a62-b7d9-dc51773e84d3 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:30:49.672072 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea9ac7-c8d5-4a62-b7d9-dc51773e84d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:30:49.691924 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-0cea9ac7-c8d5-4a62-b7d9-dc51773e84d3 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:30:49.693127 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-0cea9ac7-c8d5-4a62-b7d9-dc51773e84d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:30:49.701006 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 771/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 10:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:30:49.857195 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e86401-8932-4167-a9de-3b83e225e674 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:30:49.871622 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e86401-8932-4167-a9de-3b83e225e674 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:30:49.877300 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd75cada-cbf3-43e7-a27e-b1832d6dd04b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:30:49.877923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75cada-cbf3-43e7-a27e-b1832d6dd04b service nova] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:30:49.878233 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75cada-cbf3-43e7-a27e-b1832d6dd04b service nova] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:30:49.878507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75cada-cbf3-43e7-a27e-b1832d6dd04b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:30:49.886667 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd75cada-cbf3-43e7-a27e-b1832d6dd04b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:30:49.888322 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd75cada-cbf3-43e7-a27e-b1832d6dd04b service nova] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:30:49.897885 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 756/1527] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:30:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:30:53.837855 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e393b81-eaa1-4a4a-800f-ef84e0630198 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:30:53.838437 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e393b81-eaa1-4a4a-800f-ef84e0630198 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:30:53.838437 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e393b81-eaa1-4a4a-800f-ef84e0630198 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:30:53.839870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 772/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 10:30:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:30:53.844899 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8140b04-60d8-46a2-a63c-a75529b237d4 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:30:53.845589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8140b04-60d8-46a2-a63c-a75529b237d4 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:30:53.845693 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8140b04-60d8-46a2-a63c-a75529b237d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:30:53.869386 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-e8140b04-60d8-46a2-a63c-a75529b237d4 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:30:53.871631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-e8140b04-60d8-46a2-a63c-a75529b237d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:30:53.880621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 757/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 10:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:30:54.168954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e393b81-eaa1-4a4a-800f-ef84e0630198 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:30:54.170666 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-2e393b81-eaa1-4a4a-800f-ef84e0630198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:30:54.248913 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e393b81-eaa1-4a4a-800f-ef84e0630198 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:30:54.252502 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a062fe-03ee-4bf5-a997-269466ca5459 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:30:54.253097 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a062fe-03ee-4bf5-a997-269466ca5459 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:30:54.253375 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a062fe-03ee-4bf5-a997-269466ca5459 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:30:54.253649 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a062fe-03ee-4bf5-a997-269466ca5459 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:30:54.262441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a062fe-03ee-4bf5-a997-269466ca5459 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:30:54.263853 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a062fe-03ee-4bf5-a997-269466ca5459 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:30:54.273052 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 773/1530] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:30:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:30:56.441651 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8a3b1-cd8e-4e10-bf26-2ab7b0d8221e None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:30:56.442268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8a3b1-cd8e-4e10-bf26-2ab7b0d8221e None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:30:56.442268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8a3b1-cd8e-4e10-bf26-2ab7b0d8221e None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:30:56.443552 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 758/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 10:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:30:56.448870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec253a33-44c7-4de6-8083-720e2959c57d None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:30:56.449316 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec253a33-44c7-4de6-8083-720e2959c57d None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:30:56.449429 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec253a33-44c7-4de6-8083-720e2959c57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:30:56.468701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-ec253a33-44c7-4de6-8083-720e2959c57d None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:30:56.469894 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-ec253a33-44c7-4de6-8083-720e2959c57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:30:56.477970 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 774/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 10:30:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:30:56.767101 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd8a3b1-cd8e-4e10-bf26-2ab7b0d8221e None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:30:56.772465 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=75 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:30:56.781227 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd8a3b1-cd8e-4e10-bf26-2ab7b0d8221e None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:30:56.785965 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d4dd1-b380-4885-870a-fbcf1222965f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:30:56.786583 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d4dd1-b380-4885-870a-fbcf1222965f service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:30:56.786856 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d4dd1-b380-4885-870a-fbcf1222965f service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:30:56.787122 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d4dd1-b380-4885-870a-fbcf1222965f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:30:56.797665 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d4dd1-b380-4885-870a-fbcf1222965f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:30:56.799140 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d4dd1-b380-4885-870a-fbcf1222965f service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:30:56.808815 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 759/1533] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:30:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:31:00.594042 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f329e2b-5cc3-4123-af8a-286f82108e7c None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:31:00.594613 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f329e2b-5cc3-4123-af8a-286f82108e7c None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:31:00.594613 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f329e2b-5cc3-4123-af8a-286f82108e7c None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:31:00.596233 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 775/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 10:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:31:00.601368 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ddde0-ac29-4200-a9ff-95412aa8ece5 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:31:00.601771 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ddde0-ac29-4200-a9ff-95412aa8ece5 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:31:00.601921 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ddde0-ac29-4200-a9ff-95412aa8ece5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:31:00.621734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-aa3ddde0-ac29-4200-a9ff-95412aa8ece5 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:31:00.622990 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-aa3ddde0-ac29-4200-a9ff-95412aa8ece5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:31:00.630821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 760/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 10:31:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:31:00.940717 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f329e2b-5cc3-4123-af8a-286f82108e7c None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:31:00.942432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-4f329e2b-5cc3-4123-af8a-286f82108e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:31:01.009248 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:31:01.023392 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f329e2b-5cc3-4123-af8a-286f82108e7c None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:31:01.027589 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3eafb-38c6-4625-a557-271a75543a4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:31:01.028228 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3eafb-38c6-4625-a557-271a75543a4b service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:31:01.028504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3eafb-38c6-4625-a557-271a75543a4b service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:31:01.028776 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3eafb-38c6-4625-a557-271a75543a4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:31:01.037569 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c3eafb-38c6-4625-a557-271a75543a4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:31:01.039003 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c3eafb-38c6-4625-a557-271a75543a4b service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:31:01.048425 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 776/1536] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:31:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:31:03.205151 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760013a-eb1f-4cb8-a187-615192a9bc8e None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:31:03.205680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760013a-eb1f-4cb8-a187-615192a9bc8e None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:31:03.205680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760013a-eb1f-4cb8-a187-615192a9bc8e None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:31:03.206972 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 761/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 10:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:31:03.212266 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac391d-435a-4bb5-a1b0-e98033b65b5c None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:31:03.212677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac391d-435a-4bb5-a1b0-e98033b65b5c None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:31:03.212809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac391d-435a-4bb5-a1b0-e98033b65b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:31:03.232610 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-4dac391d-435a-4bb5-a1b0-e98033b65b5c None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:31:03.233790 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-4dac391d-435a-4bb5-a1b0-e98033b65b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:31:03.241433 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 777/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 10:31:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:31:03.545124 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760013a-eb1f-4cb8-a187-615192a9bc8e None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:31:03.546888 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-7760013a-eb1f-4cb8-a187-615192a9bc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:31:03.559662 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760013a-eb1f-4cb8-a187-615192a9bc8e None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:31:03.563088 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac19f658-3909-43d3-810f-283868fa00e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:31:03.563666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19f658-3909-43d3-810f-283868fa00e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:31:03.563969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19f658-3909-43d3-810f-283868fa00e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:31:03.564241 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19f658-3909-43d3-810f-283868fa00e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:31:03.573099 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac19f658-3909-43d3-810f-283868fa00e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:31:03.574554 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac19f658-3909-43d3-810f-283868fa00e6 service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:31:03.583785 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 762/1539] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:31:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:31:13.227501 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=11 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:31:13.579665 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=14 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:28.553197 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713eeb87-2d6e-4dc8-98cf-6a7a01f8ff89 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:28.553991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713eeb87-2d6e-4dc8-98cf-6a7a01f8ff89 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:28.553991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713eeb87-2d6e-4dc8-98cf-6a7a01f8ff89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:28.557573 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8964a-1e2c-4728-8ff6-e4673f5a4cba None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:28.558200 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8964a-1e2c-4728-8ff6-e4673f5a4cba None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:28.558200 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8964a-1e2c-4728-8ff6-e4673f5a4cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:28.596668 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-713eeb87-2d6e-4dc8-98cf-6a7a01f8ff89 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:28.598206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-713eeb87-2d6e-4dc8-98cf-6a7a01f8ff89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:28.612548 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-f5b8964a-1e2c-4728-8ff6-e4673f5a4cba None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:28.613788 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-f5b8964a-1e2c-4728-8ff6-e4673f5a4cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:28.615014 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 763/1540] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:28.621548 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe38e1-b4cb-4ed5-99d6-e93e69cf5910 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:28.621853 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe38e1-b4cb-4ed5-99d6-e93e69cf5910 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:28.622123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe38e1-b4cb-4ed5-99d6-e93e69cf5910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:28.635192 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 778/1541] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:32:28.644728 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ebbe8f-3d74-4029-95d7-8788c6ddeb3d None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:28.645069 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ebbe8f-3d74-4029-95d7-8788c6ddeb3d None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:28.645278 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ebbe8f-3d74-4029-95d7-8788c6ddeb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:28.665058 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-1cfe38e1-b4cb-4ed5-99d6-e93e69cf5910 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:28.666300 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-1cfe38e1-b4cb-4ed5-99d6-e93e69cf5910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:28.692376 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 764/1542] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:32:28.692655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-77ebbe8f-3d74-4029-95d7-8788c6ddeb3d None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:28.693500 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-77ebbe8f-3d74-4029-95d7-8788c6ddeb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:28.698296 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe38e1-b4cb-4ed5-99d6-e93e69cf5910 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:28.710397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 779/1543] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:32:28.712418 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe38e1-b4cb-4ed5-99d6-e93e69cf5910 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:28.715887 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ebbe8f-3d74-4029-95d7-8788c6ddeb3d None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:28.721538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8669c-b6f9-46cd-b2e2-2f3de23f53b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:28.722524 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8669c-b6f9-46cd-b2e2-2f3de23f53b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:28.722803 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8669c-b6f9-46cd-b2e2-2f3de23f53b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:28.723073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8669c-b6f9-46cd-b2e2-2f3de23f53b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:28.723861 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e8669c-b6f9-46cd-b2e2-2f3de23f53b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:32:28.724165 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e8669c-b6f9-46cd-b2e2-2f3de23f53b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:28.725184 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: WARNING py.warnings [None req-00e8669c-b6f9-46cd-b2e2-2f3de23f53b6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 26 10:32:28.725184 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: warnings.warn(msg) Sep 26 10:32:28.725184 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: Sep 26 10:32:28.725844 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ebbe8f-3d74-4029-95d7-8788c6ddeb3d None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:28.726092 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e8669c-b6f9-46cd-b2e2-2f3de23f53b6 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:28.740248 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 765/1544] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:32:28.741509 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4621b2b-3f5e-46a6-8858-b75559318bbe None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:28.741797 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4621b2b-3f5e-46a6-8858-b75559318bbe None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:28.742090 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4621b2b-3f5e-46a6-8858-b75559318bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:28.746694 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c97fb2-a53c-4e85-a8cb-9ab2faa98475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:28.747316 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c97fb2-a53c-4e85-a8cb-9ab2faa98475 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:28.747593 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c97fb2-a53c-4e85-a8cb-9ab2faa98475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:28.747865 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c97fb2-a53c-4e85-a8cb-9ab2faa98475 admin admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:28.748521 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c97fb2-a53c-4e85-a8cb-9ab2faa98475 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:32:28.748823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c97fb2-a53c-4e85-a8cb-9ab2faa98475 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:28.750442 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c97fb2-a53c-4e85-a8cb-9ab2faa98475 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:28.762916 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-c4621b2b-3f5e-46a6-8858-b75559318bbe None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:28.764271 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-c4621b2b-3f5e-46a6-8858-b75559318bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:28.776577 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 780/1545] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:32:28.779267 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 766/1546] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:28.779951 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4621b2b-3f5e-46a6-8858-b75559318bbe None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:28.780733 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4d04c-5814-46cc-b464-809bd5d5e1d8 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:28.781017 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4d04c-5814-46cc-b464-809bd5d5e1d8 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:28.781290 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4d04c-5814-46cc-b464-809bd5d5e1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:28.801875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4621b2b-3f5e-46a6-8858-b75559318bbe None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:28.815549 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708fcea-925c-46f5-9db5-3dea4ba449bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:28.816172 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708fcea-925c-46f5-9db5-3dea4ba449bf admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:28.816584 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708fcea-925c-46f5-9db5-3dea4ba449bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:28.816863 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708fcea-925c-46f5-9db5-3dea4ba449bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:28.817503 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e708fcea-925c-46f5-9db5-3dea4ba449bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:28.819845 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e708fcea-925c-46f5-9db5-3dea4ba449bf admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:28.839642 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-5ce4d04c-5814-46cc-b464-809bd5d5e1d8 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:28.841193 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-5ce4d04c-5814-46cc-b464-809bd5d5e1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:28.862983 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 781/1547] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:28.863814 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce4d04c-5814-46cc-b464-809bd5d5e1d8 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:28.882255 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce4d04c-5814-46cc-b464-809bd5d5e1d8 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:28.892296 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b391136-56b4-42e8-9b7c-824f328dfd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:28.892913 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b391136-56b4-42e8-9b7c-824f328dfd5b admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:28.893408 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b391136-56b4-42e8-9b7c-824f328dfd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:28.893682 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b391136-56b4-42e8-9b7c-824f328dfd5b admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:28.894351 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b391136-56b4-42e8-9b7c-824f328dfd5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:28.895103 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 767/1548] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:32:28 2022] POST /identity/v3/projects => generated 379 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:32:28.895379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: WARNING py.warnings [None req-1b391136-56b4-42e8-9b7c-824f328dfd5b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 10:32:28.895379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: warnings.warn(msg) Sep 26 10:32:28.895379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: Sep 26 10:32:28.897457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b391136-56b4-42e8-9b7c-824f328dfd5b admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:28.897587 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2f34f-5dd0-4bcb-8ae7-9e907cbc9313 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:28.897587 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2f34f-5dd0-4bcb-8ae7-9e907cbc9313 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:28.897587 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2f34f-5dd0-4bcb-8ae7-9e907cbc9313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:28.918163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-cfe2f34f-5dd0-4bcb-8ae7-9e907cbc9313 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:28.919362 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-cfe2f34f-5dd0-4bcb-8ae7-9e907cbc9313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:28.933294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 782/1549] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:32:28 2022] POST /identity/v3/projects => generated 393 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:32:28.934458 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c5e4d-3c4e-4f49-a432-c36f35953735 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:28.934765 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c5e4d-3c4e-4f49-a432-c36f35953735 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:28.935064 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c5e4d-3c4e-4f49-a432-c36f35953735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:28.954944 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-9b3c5e4d-3c4e-4f49-a432-c36f35953735 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:28.956150 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-9b3c5e4d-3c4e-4f49-a432-c36f35953735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:28.982405 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 768/1550] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:28.983674 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c802463-46d5-47e8-bddf-82f781cc2ee2 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:28.983963 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c802463-46d5-47e8-bddf-82f781cc2ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:28.984246 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c802463-46d5-47e8-bddf-82f781cc2ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.003407 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-0c802463-46d5-47e8-bddf-82f781cc2ee2 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:29.004863 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-0c802463-46d5-47e8-bddf-82f781cc2ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:29.017131 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 783/1551] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:29.018403 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795fda8-4d7a-4ade-8c56-0666c3d5e623 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:29.018684 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795fda8-4d7a-4ade-8c56-0666c3d5e623 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:29.018955 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795fda8-4d7a-4ade-8c56-0666c3d5e623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.038917 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-f795fda8-4d7a-4ade-8c56-0666c3d5e623 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:29.040109 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-f795fda8-4d7a-4ade-8c56-0666c3d5e623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:29.048035 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 784/1552] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:29.049186 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9185cf4-9ffb-4656-8498-a606f84bb6d2 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:29.049465 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9185cf4-9ffb-4656-8498-a606f84bb6d2 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:29.049733 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9185cf4-9ffb-4656-8498-a606f84bb6d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.068628 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-e9185cf4-9ffb-4656-8498-a606f84bb6d2 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:29.069750 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-e9185cf4-9ffb-4656-8498-a606f84bb6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:29.072257 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 769/1553] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:29.073383 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c802463-46d5-47e8-bddf-82f781cc2ee2 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:29.081327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 785/1554] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:29.082572 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e05e65-84bf-4454-b360-967f34e02756 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:29.082858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e05e65-84bf-4454-b360-967f34e02756 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:29.083130 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e05e65-84bf-4454-b360-967f34e02756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.091870 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c802463-46d5-47e8-bddf-82f781cc2ee2 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:29.096520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6981a55-b247-4025-8e41-a981008703db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:29.097372 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6981a55-b247-4025-8e41-a981008703db admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:29.097663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6981a55-b247-4025-8e41-a981008703db admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:29.097937 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6981a55-b247-4025-8e41-a981008703db admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.098593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6981a55-b247-4025-8e41-a981008703db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:29.099631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: WARNING py.warnings [None req-a6981a55-b247-4025-8e41-a981008703db admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 10:32:29.099631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: warnings.warn(msg) Sep 26 10:32:29.099631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: Sep 26 10:32:29.100827 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6981a55-b247-4025-8e41-a981008703db admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:29.103294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-94e05e65-84bf-4454-b360-967f34e02756 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:29.104592 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-94e05e65-84bf-4454-b360-967f34e02756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:29.112660 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 786/1555] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:29.113543 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e05e65-84bf-4454-b360-967f34e02756 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:29.123034 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e05e65-84bf-4454-b360-967f34e02756 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:29.127646 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2350cd-6450-48cd-80a9-b381a9819ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:29.128233 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2350cd-6450-48cd-80a9-b381a9819ac2 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:29.128515 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2350cd-6450-48cd-80a9-b381a9819ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:29.128788 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2350cd-6450-48cd-80a9-b381a9819ac2 admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.129419 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2350cd-6450-48cd-80a9-b381a9819ac2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:29.132719 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 770/1556] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:32:29 2022] POST /identity/v3/users => generated 290 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:32:29.132719 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6981a55-b247-4025-8e41-a981008703db admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:29.132897 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2350cd-6450-48cd-80a9-b381a9819ac2 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:29.140291 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6981a55-b247-4025-8e41-a981008703db admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:29.146990 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97021db6-1d91-468a-96bf-d23ac452888f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:29.147545 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97021db6-1d91-468a-96bf-d23ac452888f admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:29.147893 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97021db6-1d91-468a-96bf-d23ac452888f admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:29.148165 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97021db6-1d91-468a-96bf-d23ac452888f admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.148825 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97021db6-1d91-468a-96bf-d23ac452888f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:32:29.149123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97021db6-1d91-468a-96bf-d23ac452888f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:29.150770 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97021db6-1d91-468a-96bf-d23ac452888f admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:29.154178 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 787/1557] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:32:29 2022] POST /identity/v3/users => generated 297 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:32:29.155427 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf3cc3-371e-4e7a-90cf-e09d5ee769e1 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:29.155715 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf3cc3-371e-4e7a-90cf-e09d5ee769e1 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:29.155990 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf3cc3-371e-4e7a-90cf-e09d5ee769e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.164506 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 771/1558] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:32:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:32:29.165602 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97021db6-1d91-468a-96bf-d23ac452888f admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:29.174999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97021db6-1d91-468a-96bf-d23ac452888f admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:29.176978 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-21bf3cc3-371e-4e7a-90cf-e09d5ee769e1 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:29.178321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-21bf3cc3-371e-4e7a-90cf-e09d5ee769e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:29.180357 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da719c64-7e63-4559-b847-3cf2556fbf1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:29.180945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da719c64-7e63-4559-b847-3cf2556fbf1a admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:29.181223 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da719c64-7e63-4559-b847-3cf2556fbf1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:29.181494 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da719c64-7e63-4559-b847-3cf2556fbf1a admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.182764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da719c64-7e63-4559-b847-3cf2556fbf1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:32:29.183010 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da719c64-7e63-4559-b847-3cf2556fbf1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:29.184587 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da719c64-7e63-4559-b847-3cf2556fbf1a admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:29.186119 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:29.187113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 788/1559] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:29.188354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19f644-f67a-41e4-9eeb-a7128bf288f4 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:29.188641 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19f644-f67a-41e4-9eeb-a7128bf288f4 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:29.188912 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19f644-f67a-41e4-9eeb-a7128bf288f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.197609 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 772/1560] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:32:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:32:29.203373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c560ba-b838-4e62-8dca-0d9ab9e6d1bb None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:29.203771 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c560ba-b838-4e62-8dca-0d9ab9e6d1bb None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:29.204175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c560ba-b838-4e62-8dca-0d9ab9e6d1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.209370 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-2d19f644-f67a-41e4-9eeb-a7128bf288f4 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:29.210596 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-2d19f644-f67a-41e4-9eeb-a7128bf288f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:29.223424 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 789/1561] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:29.225015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19bcdc-a898-4057-be2c-2a34a1312baa None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:29.225311 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19bcdc-a898-4057-be2c-2a34a1312baa None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:29.225592 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19bcdc-a898-4057-be2c-2a34a1312baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.226587 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-68c560ba-b838-4e62-8dca-0d9ab9e6d1bb None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:29.228025 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-68c560ba-b838-4e62-8dca-0d9ab9e6d1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:29.236197 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 773/1562] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:29.237231 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c560ba-b838-4e62-8dca-0d9ab9e6d1bb None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:29.246240 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-5f19bcdc-a898-4057-be2c-2a34a1312baa None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:29.247220 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c560ba-b838-4e62-8dca-0d9ab9e6d1bb None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:29.247445 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-5f19bcdc-a898-4057-be2c-2a34a1312baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:29.251813 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2c242-d6be-4b86-8ed6-79558538563e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:29.252390 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2c242-d6be-4b86-8ed6-79558538563e admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:29.252667 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2c242-d6be-4b86-8ed6-79558538563e admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:29.252937 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2c242-d6be-4b86-8ed6-79558538563e admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.253568 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f2c242-d6be-4b86-8ed6-79558538563e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:29.255179 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f2c242-d6be-4b86-8ed6-79558538563e admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:29.255708 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 790/1563] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:29.256490 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f19bcdc-a898-4057-be2c-2a34a1312baa None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:29.266872 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f19bcdc-a898-4057-be2c-2a34a1312baa None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:29.268499 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 774/1564] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:32:29.269368 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2c242-d6be-4b86-8ed6-79558538563e admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:29.273134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e270001-aa86-4ca0-bfd5-d9741913764b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:29.273731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e270001-aa86-4ca0-bfd5-d9741913764b admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:29.274072 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e270001-aa86-4ca0-bfd5-d9741913764b admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:29.274354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e270001-aa86-4ca0-bfd5-d9741913764b admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.274990 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e270001-aa86-4ca0-bfd5-d9741913764b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:29.276610 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e270001-aa86-4ca0-bfd5-d9741913764b admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:29.279068 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2c242-d6be-4b86-8ed6-79558538563e admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:29.283762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07924b43-d95c-4755-ba2d-da5463671a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:29.284415 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07924b43-d95c-4755-ba2d-da5463671a16 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:29.284693 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07924b43-d95c-4755-ba2d-da5463671a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:29.284974 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07924b43-d95c-4755-ba2d-da5463671a16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.285575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07924b43-d95c-4755-ba2d-da5463671a16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:29.289591 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 791/1565] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:32:29.289745 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07924b43-d95c-4755-ba2d-da5463671a16 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:29.289852 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e270001-aa86-4ca0-bfd5-d9741913764b admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:29.297299 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e270001-aa86-4ca0-bfd5-d9741913764b admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:29.301871 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c41f84-444c-48cd-88f5-27e33181894b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:29.302473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c41f84-444c-48cd-88f5-27e33181894b admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:29.302888 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c41f84-444c-48cd-88f5-27e33181894b admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:29.303172 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c41f84-444c-48cd-88f5-27e33181894b admin admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.303810 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c41f84-444c-48cd-88f5-27e33181894b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:32:29.304117 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c41f84-444c-48cd-88f5-27e33181894b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:29.305675 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c41f84-444c-48cd-88f5-27e33181894b admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:29.318244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 775/1566] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:32:29 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:32:29.318244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07924b43-d95c-4755-ba2d-da5463671a16 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:29.329570 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 792/1567] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:32:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:32:29.329570 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c41f84-444c-48cd-88f5-27e33181894b admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:29.333479 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c41f84-444c-48cd-88f5-27e33181894b admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:29.354129 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:29.381207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07924b43-d95c-4755-ba2d-da5463671a16 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:29.386515 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0f704-2949-4002-a42d-0fc3504d5b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:29.387120 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0f704-2949-4002-a42d-0fc3504d5b37 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:29.387462 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0f704-2949-4002-a42d-0fc3504d5b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:29.387670 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0f704-2949-4002-a42d-0fc3504d5b37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.388298 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db0f704-2949-4002-a42d-0fc3504d5b37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:29.390224 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db0f704-2949-4002-a42d-0fc3504d5b37 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:29.394704 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244692f7-6dae-4b46-bc41-1784aaa7b60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:29.395354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244692f7-6dae-4b46-bc41-1784aaa7b60a admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:29.395634 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244692f7-6dae-4b46-bc41-1784aaa7b60a admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:29.395902 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244692f7-6dae-4b46-bc41-1784aaa7b60a admin admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.396565 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244692f7-6dae-4b46-bc41-1784aaa7b60a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:32:29.396860 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244692f7-6dae-4b46-bc41-1784aaa7b60a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:29.398516 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244692f7-6dae-4b46-bc41-1784aaa7b60a admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:29.417832 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 793/1568] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:32:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:32:29.417832 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244692f7-6dae-4b46-bc41-1784aaa7b60a admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:29.418171 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 776/1569] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:32:29 2022] POST /identity/v3/projects => generated 405 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:32:29.418638 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0f704-2949-4002-a42d-0fc3504d5b37 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:29.426706 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244692f7-6dae-4b46-bc41-1784aaa7b60a admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:29.480062 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0f704-2949-4002-a42d-0fc3504d5b37 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:29.485130 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc568c93-04f1-433a-9fde-d91e0ffff8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:29.485744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc568c93-04f1-433a-9fde-d91e0ffff8b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:29.486046 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc568c93-04f1-433a-9fde-d91e0ffff8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:29.486165 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684c264-4218-4eaf-9df6-710320d357fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:29.486401 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc568c93-04f1-433a-9fde-d91e0ffff8b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.486765 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684c264-4218-4eaf-9df6-710320d357fa admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:29.486961 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc568c93-04f1-433a-9fde-d91e0ffff8b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:32:29.487104 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684c264-4218-4eaf-9df6-710320d357fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:29.487255 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc568c93-04f1-433a-9fde-d91e0ffff8b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:29.487555 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684c264-4218-4eaf-9df6-710320d357fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.488471 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684c264-4218-4eaf-9df6-710320d357fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:32:29.488817 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684c264-4218-4eaf-9df6-710320d357fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:29.488959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc568c93-04f1-433a-9fde-d91e0ffff8b7 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:29.490746 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684c264-4218-4eaf-9df6-710320d357fa admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:29.501916 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 777/1570] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:32:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:32:29.502948 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc568c93-04f1-433a-9fde-d91e0ffff8b7 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:29.511637 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 794/1571] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:32:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:32:29.512381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684c264-4218-4eaf-9df6-710320d357fa admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:29.520595 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc568c93-04f1-433a-9fde-d91e0ffff8b7 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:29.522370 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684c264-4218-4eaf-9df6-710320d357fa admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:29.525308 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d17a6f-db77-453a-ac98-846e48aacf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:29.525931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d17a6f-db77-453a-ac98-846e48aacf9a admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:29.526281 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d17a6f-db77-453a-ac98-846e48aacf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:29.526558 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d17a6f-db77-453a-ac98-846e48aacf9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.527058 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c6be9-7784-4843-90a9-2bbde7e7e52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:29.527185 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d17a6f-db77-453a-ac98-846e48aacf9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:29.527639 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c6be9-7784-4843-90a9-2bbde7e7e52c admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:29.527922 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c6be9-7784-4843-90a9-2bbde7e7e52c admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:29.528196 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c6be9-7784-4843-90a9-2bbde7e7e52c admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.528806 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d17a6f-db77-453a-ac98-846e48aacf9a admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:29.528908 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c6be9-7784-4843-90a9-2bbde7e7e52c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:29.530614 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c6be9-7784-4843-90a9-2bbde7e7e52c admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:29.539460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 778/1572] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:32:29.543817 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 795/1573] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:32:29.543981 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d17a6f-db77-453a-ac98-846e48aacf9a admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:29.544071 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c6be9-7784-4843-90a9-2bbde7e7e52c admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:29.551322 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c6be9-7784-4843-90a9-2bbde7e7e52c admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:29.553402 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d17a6f-db77-453a-ac98-846e48aacf9a admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:29.556843 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1a476-fdea-4d28-9546-73e46fefb1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:29.558049 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1a476-fdea-4d28-9546-73e46fefb1a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:29.558568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1a476-fdea-4d28-9546-73e46fefb1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:29.558673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1a476-fdea-4d28-9546-73e46fefb1a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.560929 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae1a476-fdea-4d28-9546-73e46fefb1a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:29.561045 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05faee2c-da91-4ddc-a421-42a2943171b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:29.561045 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05faee2c-da91-4ddc-a421-42a2943171b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:29.561045 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05faee2c-da91-4ddc-a421-42a2943171b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:29.561045 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05faee2c-da91-4ddc-a421-42a2943171b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.561455 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae1a476-fdea-4d28-9546-73e46fefb1a1 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:29.561927 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05faee2c-da91-4ddc-a421-42a2943171b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:29.563905 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05faee2c-da91-4ddc-a421-42a2943171b5 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:29.589532 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 779/1574] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:32:29 2022] POST /identity/v3/users => generated 289 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:32:29.590725 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05faee2c-da91-4ddc-a421-42a2943171b5 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:29.595049 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 796/1575] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:32:29 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:32:29.595741 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1a476-fdea-4d28-9546-73e46fefb1a1 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:29.599098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:29.603091 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05faee2c-da91-4ddc-a421-42a2943171b5 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:29.660181 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d5c42-1a06-46c8-b0c1-2199aab5a3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:29.660440 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1a476-fdea-4d28-9546-73e46fefb1a1 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:29.660792 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d5c42-1a06-46c8-b0c1-2199aab5a3bf admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:29.661077 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d5c42-1a06-46c8-b0c1-2199aab5a3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:29.661353 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d5c42-1a06-46c8-b0c1-2199aab5a3bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.661982 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d5c42-1a06-46c8-b0c1-2199aab5a3bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:29.663940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d5c42-1a06-46c8-b0c1-2199aab5a3bf admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:29.665676 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf244cc-8bd3-41ae-8cf6-f67eeeb14c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:29.666313 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf244cc-8bd3-41ae-8cf6-f67eeeb14c63 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:29.667101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf244cc-8bd3-41ae-8cf6-f67eeeb14c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:29.667366 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf244cc-8bd3-41ae-8cf6-f67eeeb14c63 admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.667990 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf244cc-8bd3-41ae-8cf6-f67eeeb14c63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:29.669869 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf244cc-8bd3-41ae-8cf6-f67eeeb14c63 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:29.692389 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 797/1576] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:32:29 2022] POST /identity/v3/users => generated 303 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:32:29.693027 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf244cc-8bd3-41ae-8cf6-f67eeeb14c63 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:29.701678 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 780/1577] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:32:29 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:32:29.702546 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d5c42-1a06-46c8-b0c1-2199aab5a3bf admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:29.703419 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf244cc-8bd3-41ae-8cf6-f67eeeb14c63 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:29.709293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:29.764270 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ae384-1e8f-4df0-86fd-30b6d799796b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:29.764878 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ae384-1e8f-4df0-86fd-30b6d799796b admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:29.765154 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ae384-1e8f-4df0-86fd-30b6d799796b admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:29.765481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ae384-1e8f-4df0-86fd-30b6d799796b admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.766140 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ae384-1e8f-4df0-86fd-30b6d799796b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:29.767949 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d5c42-1a06-46c8-b0c1-2199aab5a3bf admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:29.768063 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ae384-1e8f-4df0-86fd-30b6d799796b admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:29.772537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311a243-ce34-444d-81e0-d39515c2af2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:29.773135 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311a243-ce34-444d-81e0-d39515c2af2c admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:29.773422 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311a243-ce34-444d-81e0-d39515c2af2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:29.773690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311a243-ce34-444d-81e0-d39515c2af2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.774352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d311a243-ce34-444d-81e0-d39515c2af2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:29.776244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d311a243-ce34-444d-81e0-d39515c2af2c admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:29.797614 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 798/1578] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:32:29 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:32:29.801969 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ae384-1e8f-4df0-86fd-30b6d799796b admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:29.804636 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 781/1579] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:32:29 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:32:29.805737 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311a243-ce34-444d-81e0-d39515c2af2c admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:29.865408 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ae384-1e8f-4df0-86fd-30b6d799796b admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:29.869245 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311a243-ce34-444d-81e0-d39515c2af2c admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:29.870578 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735936dc-0582-4c0e-ac69-065fec39abbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:29.871233 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735936dc-0582-4c0e-ac69-065fec39abbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:29.871509 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735936dc-0582-4c0e-ac69-065fec39abbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:29.871780 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735936dc-0582-4c0e-ac69-065fec39abbe admin admin] PATH_INFO: `/v3/projects/c64a07f1fc58419e8dbd23d28a43e98b/users/398f3e872ad9499fbb26cc48ca9544d8/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.874341 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370d42b-a6f6-4bd0-b6c4-64ac2d07a190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:29.875085 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370d42b-a6f6-4bd0-b6c4-64ac2d07a190 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:29.875356 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370d42b-a6f6-4bd0-b6c4-64ac2d07a190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:29.875624 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370d42b-a6f6-4bd0-b6c4-64ac2d07a190 admin admin] PATH_INFO: `/v3/projects/615e5825389c4f1d96e6398bf1dab735/users/5085a918c6b74488b3dd70c1fb257381/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.886643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735936dc-0582-4c0e-ac69-065fec39abbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=c64a07f1fc58419e8dbd23d28a43e98b, user_id=398f3e872ad9499fbb26cc48ca9544d8, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:29.887941 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: WARNING py.warnings [None req-735936dc-0582-4c0e-ac69-065fec39abbe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 10:32:29.887941 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: warnings.warn(msg) Sep 26 10:32:29.887941 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: Sep 26 10:32:29.889616 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735936dc-0582-4c0e-ac69-065fec39abbe admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:29.890613 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a370d42b-a6f6-4bd0-b6c4-64ac2d07a190 admin admin] RBAC: Authorizing `identity:create_grant(project_id=615e5825389c4f1d96e6398bf1dab735, user_id=5085a918c6b74488b3dd70c1fb257381, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:29.893278 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a370d42b-a6f6-4bd0-b6c4-64ac2d07a190 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:29.902016 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:29.903130 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 799/1580] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:32:29 2022] PUT /identity/v3/projects/c64a07f1fc58419e8dbd23d28a43e98b/users/398f3e872ad9499fbb26cc48ca9544d8/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:29.907300 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735936dc-0582-4c0e-ac69-065fec39abbe admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:29.909717 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 782/1581] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:32:29 2022] PUT /identity/v3/projects/615e5825389c4f1d96e6398bf1dab735/users/5085a918c6b74488b3dd70c1fb257381/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:29.910589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370d42b-a6f6-4bd0-b6c4-64ac2d07a190 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:29.970659 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735936dc-0582-4c0e-ac69-065fec39abbe admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:29.973758 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370d42b-a6f6-4bd0-b6c4-64ac2d07a190 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:29.975723 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbb4c7-8280-46db-bb49-a838ac5e6a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:29.976332 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbb4c7-8280-46db-bb49-a838ac5e6a60 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:29.976613 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbb4c7-8280-46db-bb49-a838ac5e6a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:29.976889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbb4c7-8280-46db-bb49-a838ac5e6a60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.977534 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfbb4c7-8280-46db-bb49-a838ac5e6a60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:29.978342 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890f9f6-51a5-4700-adb2-6a087a4b71de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:29.978935 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890f9f6-51a5-4700-adb2-6a087a4b71de admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:29.979751 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfbb4c7-8280-46db-bb49-a838ac5e6a60 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:29.979868 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890f9f6-51a5-4700-adb2-6a087a4b71de admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:29.980022 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890f9f6-51a5-4700-adb2-6a087a4b71de admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:29.980677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f890f9f6-51a5-4700-adb2-6a087a4b71de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:29.982587 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f890f9f6-51a5-4700-adb2-6a087a4b71de admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:29.989786 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 800/1582] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:32:29.990636 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbb4c7-8280-46db-bb49-a838ac5e6a60 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:30.000975 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbb4c7-8280-46db-bb49-a838ac5e6a60 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:30.004079 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 783/1583] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:32:29 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:32:30.004860 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890f9f6-51a5-4700-adb2-6a087a4b71de admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:30.008591 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9e7cd-00fe-473c-a4f5-fe7d55cc0237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:30.009190 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9e7cd-00fe-473c-a4f5-fe7d55cc0237 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:30.009471 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9e7cd-00fe-473c-a4f5-fe7d55cc0237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:30.009803 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9e7cd-00fe-473c-a4f5-fe7d55cc0237 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:30.010453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca9e7cd-00fe-473c-a4f5-fe7d55cc0237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:30.012044 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca9e7cd-00fe-473c-a4f5-fe7d55cc0237 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:30.014746 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890f9f6-51a5-4700-adb2-6a087a4b71de admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:30.019598 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d4688-dad7-455d-8041-3d7e3743ecc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:30.020178 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d4688-dad7-455d-8041-3d7e3743ecc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:30.020458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d4688-dad7-455d-8041-3d7e3743ecc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:30.020730 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d4688-dad7-455d-8041-3d7e3743ecc8 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:30.021347 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d4688-dad7-455d-8041-3d7e3743ecc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:30.022369 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 801/1584] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:32:30.023105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9e7cd-00fe-473c-a4f5-fe7d55cc0237 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:30.023277 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d4688-dad7-455d-8041-3d7e3743ecc8 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:30.028622 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:30.035146 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9e7cd-00fe-473c-a4f5-fe7d55cc0237 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:30.041499 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801af91d-d175-48a6-9daf-30002628b97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:30.042855 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801af91d-d175-48a6-9daf-30002628b97a admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:30.043142 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801af91d-d175-48a6-9daf-30002628b97a admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:30.043417 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801af91d-d175-48a6-9daf-30002628b97a admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:30.044041 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801af91d-d175-48a6-9daf-30002628b97a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:30.045954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801af91d-d175-48a6-9daf-30002628b97a admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:30.046721 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 784/1585] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:32:30 2022] POST /identity/v3/users => generated 287 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:32:30.047511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d4688-dad7-455d-8041-3d7e3743ecc8 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:30.057201 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d4688-dad7-455d-8041-3d7e3743ecc8 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:30.063047 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214012f-bc8e-43ac-873a-9f5c93680188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:30.063634 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214012f-bc8e-43ac-873a-9f5c93680188 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:30.063918 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214012f-bc8e-43ac-873a-9f5c93680188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:30.064193 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214012f-bc8e-43ac-873a-9f5c93680188 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:30.064812 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9214012f-bc8e-43ac-873a-9f5c93680188 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:30.066792 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9214012f-bc8e-43ac-873a-9f5c93680188 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:30.069371 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 802/1586] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:32:30 2022] POST /identity/v3/users => generated 292 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:32:30.070500 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5855e22-0906-456a-a0c1-c1e0deefe103 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:30.070784 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5855e22-0906-456a-a0c1-c1e0deefe103 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:30.071061 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5855e22-0906-456a-a0c1-c1e0deefe103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:30.089821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 785/1587] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:32:30 2022] POST /identity/v3/users => generated 291 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:32:30.090506 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-d5855e22-0906-456a-a0c1-c1e0deefe103 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:30.091101 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcfb03-a20e-44aa-a461-1306961781ff None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:30.091377 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcfb03-a20e-44aa-a461-1306961781ff None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:30.091652 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcfb03-a20e-44aa-a461-1306961781ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:30.091774 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-d5855e22-0906-456a-a0c1-c1e0deefe103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:30.100093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 803/1588] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:30.100880 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5855e22-0906-456a-a0c1-c1e0deefe103 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:30.110895 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5855e22-0906-456a-a0c1-c1e0deefe103 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:30.112515 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-8dfcfb03-a20e-44aa-a461-1306961781ff None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:30.113964 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-8dfcfb03-a20e-44aa-a461-1306961781ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:30.115537 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f8545-51db-4cad-844f-f04ea4f12e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:30.116071 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f8545-51db-4cad-844f-f04ea4f12e48 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:30.116366 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f8545-51db-4cad-844f-f04ea4f12e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:30.116633 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f8545-51db-4cad-844f-f04ea4f12e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:30.117264 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f8545-51db-4cad-844f-f04ea4f12e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:30.118879 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f8545-51db-4cad-844f-f04ea4f12e48 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:30.122103 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 786/1589] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:30.125305 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfcfb03-a20e-44aa-a461-1306961781ff None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:30.129037 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 804/1590] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:32:30.129834 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f8545-51db-4cad-844f-f04ea4f12e48 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:30.135384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfcfb03-a20e-44aa-a461-1306961781ff None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:30.139304 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f8545-51db-4cad-844f-f04ea4f12e48 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:30.140232 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d0f0b-43b5-49a0-bce7-979f67dcf2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:30.140816 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d0f0b-43b5-49a0-bce7-979f67dcf2a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:30.141087 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d0f0b-43b5-49a0-bce7-979f67dcf2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:30.141355 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d0f0b-43b5-49a0-bce7-979f67dcf2a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:30.141989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d0f0b-43b5-49a0-bce7-979f67dcf2a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:30.143612 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d0f0b-43b5-49a0-bce7-979f67dcf2a5 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:30.143778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e21d6e-2acc-401e-8448-4a4f7da65cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:30.144375 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e21d6e-2acc-401e-8448-4a4f7da65cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:30.144650 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e21d6e-2acc-401e-8448-4a4f7da65cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:30.144919 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e21d6e-2acc-401e-8448-4a4f7da65cd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:30.145534 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e21d6e-2acc-401e-8448-4a4f7da65cd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:30.147124 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e21d6e-2acc-401e-8448-4a4f7da65cd9 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:30.154094 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 787/1591] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:32:30.154865 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d0f0b-43b5-49a0-bce7-979f67dcf2a5 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:30.160419 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 805/1592] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:32:30.160419 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e21d6e-2acc-401e-8448-4a4f7da65cd9 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:30.164606 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d0f0b-43b5-49a0-bce7-979f67dcf2a5 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:30.168263 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e21d6e-2acc-401e-8448-4a4f7da65cd9 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:30.169780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dda957-6da4-4fda-8450-e80936acc6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:30.170384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dda957-6da4-4fda-8450-e80936acc6af admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:30.170656 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dda957-6da4-4fda-8450-e80936acc6af admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:30.170929 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dda957-6da4-4fda-8450-e80936acc6af admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:30.171572 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dda957-6da4-4fda-8450-e80936acc6af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:30.173114 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dda957-6da4-4fda-8450-e80936acc6af admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:30.173314 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d87fb-b640-4390-aa38-35b52242a722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:30.173927 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d87fb-b640-4390-aa38-35b52242a722 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:30.174255 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d87fb-b640-4390-aa38-35b52242a722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:30.174511 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d87fb-b640-4390-aa38-35b52242a722 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:30.175144 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d87fb-b640-4390-aa38-35b52242a722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:30.176903 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d87fb-b640-4390-aa38-35b52242a722 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:30.183149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 788/1593] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:32:30.183927 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dda957-6da4-4fda-8450-e80936acc6af admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:30.187600 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 806/1594] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:32:30.188442 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d87fb-b640-4390-aa38-35b52242a722 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:30.191819 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-ca3d87fb-b640-4390-aa38-35b52242a722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:30.195269 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dda957-6da4-4fda-8450-e80936acc6af admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:30.199774 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20cc3c-4708-40b5-9a7f-753470820ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:30.200346 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20cc3c-4708-40b5-9a7f-753470820ca9 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:30.200622 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20cc3c-4708-40b5-9a7f-753470820ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:30.200918 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20cc3c-4708-40b5-9a7f-753470820ca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:30.201538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da20cc3c-4708-40b5-9a7f-753470820ca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:30.203123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da20cc3c-4708-40b5-9a7f-753470820ca9 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:30.213052 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 789/1595] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:32:30.214349 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20cc3c-4708-40b5-9a7f-753470820ca9 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:30.215901 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-da20cc3c-4708-40b5-9a7f-753470820ca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:30.248775 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d87fb-b640-4390-aa38-35b52242a722 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:30.252419 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021ff09-d92d-41f3-9336-25ea44abdbaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:30.253018 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021ff09-d92d-41f3-9336-25ea44abdbaa service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:30.253294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021ff09-d92d-41f3-9336-25ea44abdbaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:30.253566 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021ff09-d92d-41f3-9336-25ea44abdbaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:30.262485 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4021ff09-d92d-41f3-9336-25ea44abdbaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:30.263896 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4021ff09-d92d-41f3-9336-25ea44abdbaa service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:30.272673 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20cc3c-4708-40b5-9a7f-753470820ca9 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:30.273268 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 807/1596] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:32:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:32:30.274155 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021ff09-d92d-41f3-9336-25ea44abdbaa service neutron] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:30.277229 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe4968-e3f0-4e9a-90b2-01bedc4611f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:30.277830 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe4968-e3f0-4e9a-90b2-01bedc4611f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:30.278270 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe4968-e3f0-4e9a-90b2-01bedc4611f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:30.278549 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe4968-e3f0-4e9a-90b2-01bedc4611f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:30.284304 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021ff09-d92d-41f3-9336-25ea44abdbaa service neutron] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:30.288496 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe4968-e3f0-4e9a-90b2-01bedc4611f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:30.288901 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ac41e-a3f6-4f2b-be5d-b9a85ca0cb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:30.289674 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ac41e-a3f6-4f2b-be5d-b9a85ca0cb5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:30.289960 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ac41e-a3f6-4f2b-be5d-b9a85ca0cb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:30.290087 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe4968-e3f0-4e9a-90b2-01bedc4611f4 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:30.290901 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ac41e-a3f6-4f2b-be5d-b9a85ca0cb5c admin admin] PATH_INFO: `/v3/projects/3c219db8370d4ef49f235020868144d7/users/cdcfab461e534465a906c0eef201e423/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:30.302852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 790/1597] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:32:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:32:30.303581 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe4968-e3f0-4e9a-90b2-01bedc4611f4 service neutron] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:30.307249 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ac41e-a3f6-4f2b-be5d-b9a85ca0cb5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c219db8370d4ef49f235020868144d7, user_id=cdcfab461e534465a906c0eef201e423, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:30.310081 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ac41e-a3f6-4f2b-be5d-b9a85ca0cb5c admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:30.314249 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe4968-e3f0-4e9a-90b2-01bedc4611f4 service neutron] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:30.319741 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760c9ba-21a2-47e8-b571-7a0ce58549e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:30.320506 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760c9ba-21a2-47e8-b571-7a0ce58549e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:30.320790 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760c9ba-21a2-47e8-b571-7a0ce58549e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:30.321122 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760c9ba-21a2-47e8-b571-7a0ce58549e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:30.321741 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5760c9ba-21a2-47e8-b571-7a0ce58549e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:30.323991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5760c9ba-21a2-47e8-b571-7a0ce58549e2 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:30.325640 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 808/1598] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:32:30 2022] PUT /identity/v3/projects/3c219db8370d4ef49f235020868144d7/users/cdcfab461e534465a906c0eef201e423/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:30.326495 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ac41e-a3f6-4f2b-be5d-b9a85ca0cb5c admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:30.336724 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 791/1599] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:32:30.339670 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760c9ba-21a2-47e8-b571-7a0ce58549e2 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:30.346478 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:30.351566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760c9ba-21a2-47e8-b571-7a0ce58549e2 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:30.407288 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ac41e-a3f6-4f2b-be5d-b9a85ca0cb5c admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:30.412925 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a224ea5-5c19-4590-92cd-003808af4410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:30.413677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a224ea5-5c19-4590-92cd-003808af4410 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:30.414053 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a224ea5-5c19-4590-92cd-003808af4410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:30.414412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a224ea5-5c19-4590-92cd-003808af4410 admin admin] PATH_INFO: `/v3/projects/34ab3676c5a24990a49c71f0e5055105/users/d06fcd37160142ecaf45f9605416aace/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:30.419261 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f9562-af4d-4dd7-b187-e51fe2298d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:30.419880 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f9562-af4d-4dd7-b187-e51fe2298d1a admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:30.420214 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f9562-af4d-4dd7-b187-e51fe2298d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:30.420436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f9562-af4d-4dd7-b187-e51fe2298d1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:30.421103 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933f9562-af4d-4dd7-b187-e51fe2298d1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:30.422882 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933f9562-af4d-4dd7-b187-e51fe2298d1a admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:30.434155 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a224ea5-5c19-4590-92cd-003808af4410 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34ab3676c5a24990a49c71f0e5055105, user_id=d06fcd37160142ecaf45f9605416aace, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:30.435412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 792/1600] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:32:30.436233 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f9562-af4d-4dd7-b187-e51fe2298d1a admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:30.437336 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a224ea5-5c19-4590-92cd-003808af4410 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:30.448681 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f9562-af4d-4dd7-b187-e51fe2298d1a admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:30.452811 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 809/1601] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:32:30 2022] PUT /identity/v3/projects/34ab3676c5a24990a49c71f0e5055105/users/d06fcd37160142ecaf45f9605416aace/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:30.453456 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a224ea5-5c19-4590-92cd-003808af4410 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:30.470242 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:30.525141 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee95298d-2eff-420a-a5b9-c0c92a740f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:30.525844 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95298d-2eff-420a-a5b9-c0c92a740f88 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:30.526789 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95298d-2eff-420a-a5b9-c0c92a740f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:30.527128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95298d-2eff-420a-a5b9-c0c92a740f88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:30.528513 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee95298d-2eff-420a-a5b9-c0c92a740f88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:30.530479 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee95298d-2eff-420a-a5b9-c0c92a740f88 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:30.536498 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a224ea5-5c19-4590-92cd-003808af4410 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:30.541945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7917fae-00f7-449b-9684-8a2d7c1dc854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:30.545554 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7917fae-00f7-449b-9684-8a2d7c1dc854 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:30.545703 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 793/1602] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:32:30.545839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7917fae-00f7-449b-9684-8a2d7c1dc854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:30.545839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7917fae-00f7-449b-9684-8a2d7c1dc854 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:30.545839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7917fae-00f7-449b-9684-8a2d7c1dc854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:30.546144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94028521-32d6-46be-8390-3c2e19a1d602 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:30.546144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94028521-32d6-46be-8390-3c2e19a1d602 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:30.546144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94028521-32d6-46be-8390-3c2e19a1d602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:30.546606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7917fae-00f7-449b-9684-8a2d7c1dc854 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:30.559108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 810/1603] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:32:30.559938 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7917fae-00f7-449b-9684-8a2d7c1dc854 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:30.568458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-94028521-32d6-46be-8390-3c2e19a1d602 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:30.569942 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-94028521-32d6-46be-8390-3c2e19a1d602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:30.573600 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7917fae-00f7-449b-9684-8a2d7c1dc854 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:30.578831 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284309b2-7fb9-416f-a111-a963d830b745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:30.579538 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284309b2-7fb9-416f-a111-a963d830b745 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:30.579816 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284309b2-7fb9-416f-a111-a963d830b745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:30.580308 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284309b2-7fb9-416f-a111-a963d830b745 admin admin] PATH_INFO: `/v3/projects/278d2e73018e464ab63524b9e4ebd32b/users/33953f2fd2f2469e8668120ea4f9bc92/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:30.582192 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 794/1604] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:30.583811 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94028521-32d6-46be-8390-3c2e19a1d602 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:30.597108 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94028521-32d6-46be-8390-3c2e19a1d602 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:30.600030 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284309b2-7fb9-416f-a111-a963d830b745 admin admin] RBAC: Authorizing `identity:create_grant(project_id=278d2e73018e464ab63524b9e4ebd32b, user_id=33953f2fd2f2469e8668120ea4f9bc92, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:30.602712 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284309b2-7fb9-416f-a111-a963d830b745 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:30.603175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1485adbf-a00a-4fa2-9a1f-c939004b3777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:30.604006 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485adbf-a00a-4fa2-9a1f-c939004b3777 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:30.604161 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485adbf-a00a-4fa2-9a1f-c939004b3777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:30.604525 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485adbf-a00a-4fa2-9a1f-c939004b3777 admin admin] PATH_INFO: `/v3/projects/8472411865af4f63b0d54d57ce2aa31a/users/4099fa631da548508b1a196887a9160e/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:30.619633 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 811/1605] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:32:30 2022] PUT /identity/v3/projects/278d2e73018e464ab63524b9e4ebd32b/users/33953f2fd2f2469e8668120ea4f9bc92/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:30.620977 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762d03f-0171-4e53-9480-fa3987506f8b None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:30.621264 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762d03f-0171-4e53-9480-fa3987506f8b None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:30.621550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762d03f-0171-4e53-9480-fa3987506f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:30.630092 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1485adbf-a00a-4fa2-9a1f-c939004b3777 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8472411865af4f63b0d54d57ce2aa31a, user_id=4099fa631da548508b1a196887a9160e, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:30.633269 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1485adbf-a00a-4fa2-9a1f-c939004b3777 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:30.649188 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 795/1606] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:32:30 2022] PUT /identity/v3/projects/8472411865af4f63b0d54d57ce2aa31a/users/4099fa631da548508b1a196887a9160e/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:30.650407 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1485adbf-a00a-4fa2-9a1f-c939004b3777 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:30.653216 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-9762d03f-0171-4e53-9480-fa3987506f8b None None] MFA Rules not processed for user `d06fcd37160142ecaf45f9605416aace`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:30.654672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-9762d03f-0171-4e53-9480-fa3987506f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:30.721169 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1485adbf-a00a-4fa2-9a1f-c939004b3777 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:30.726211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94cd90-cba3-49bc-8832-e5296543f246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:30.726898 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94cd90-cba3-49bc-8832-e5296543f246 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:30.727183 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94cd90-cba3-49bc-8832-e5296543f246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:30.727499 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94cd90-cba3-49bc-8832-e5296543f246 admin admin] PATH_INFO: `/v3/projects/49df2764440f4f2cb68d8d6284f07f47/users/b486a094550541729c19c8713a648243/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:30.740364 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 812/1607] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:30 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:30.744398 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762d03f-0171-4e53-9480-fa3987506f8b None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:30.745617 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:30.751368 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94cd90-cba3-49bc-8832-e5296543f246 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49df2764440f4f2cb68d8d6284f07f47, user_id=b486a094550541729c19c8713a648243, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:30.754520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762d03f-0171-4e53-9480-fa3987506f8b None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:30.754958 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94cd90-cba3-49bc-8832-e5296543f246 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:30.760110 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4373ee3-5c97-4fb8-8b43-f55a580e22c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:30.762379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4373ee3-5c97-4fb8-8b43-f55a580e22c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:30.762829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4373ee3-5c97-4fb8-8b43-f55a580e22c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:30.763113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4373ee3-5c97-4fb8-8b43-f55a580e22c7 admin admin] PATH_INFO: `/v3/projects/be07abf31b0b49ebb6fd8ce42e1583f2/users/8545e58f849249dca7f95e68295a3674/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:30.774462 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 796/1608] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:32:30 2022] PUT /identity/v3/projects/49df2764440f4f2cb68d8d6284f07f47/users/b486a094550541729c19c8713a648243/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:30.774462 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94cd90-cba3-49bc-8832-e5296543f246 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:30.783778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4373ee3-5c97-4fb8-8b43-f55a580e22c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be07abf31b0b49ebb6fd8ce42e1583f2, user_id=8545e58f849249dca7f95e68295a3674, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:30.787007 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4373ee3-5c97-4fb8-8b43-f55a580e22c7 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:30.818638 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 813/1609] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:32:30 2022] PUT /identity/v3/projects/be07abf31b0b49ebb6fd8ce42e1583f2/users/8545e58f849249dca7f95e68295a3674/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:30.825323 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83224fcc-4fe9-4adf-b0a7-dc8a6eba25c7 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:30.825961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83224fcc-4fe9-4adf-b0a7-dc8a6eba25c7 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:30.828084 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83224fcc-4fe9-4adf-b0a7-dc8a6eba25c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:30.858683 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94cd90-cba3-49bc-8832-e5296543f246 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:30.882078 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-83224fcc-4fe9-4adf-b0a7-dc8a6eba25c7 None None] MFA Rules not processed for user `33953f2fd2f2469e8668120ea4f9bc92`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:30.899699 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-83224fcc-4fe9-4adf-b0a7-dc8a6eba25c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:30.922749 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f152d2-2364-4b95-aaf9-1a18c2c26d1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:30.923582 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f152d2-2364-4b95-aaf9-1a18c2c26d1d service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:30.924022 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f152d2-2364-4b95-aaf9-1a18c2c26d1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:30.924300 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f152d2-2364-4b95-aaf9-1a18c2c26d1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:30.972416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:31.026857 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 814/1610] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:31.026857 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98902331-9ea3-402e-b2f0-00448cbd8525 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:31.026857 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98902331-9ea3-402e-b2f0-00448cbd8525 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:31.026857 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98902331-9ea3-402e-b2f0-00448cbd8525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:31.063095 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f152d2-2364-4b95-aaf9-1a18c2c26d1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:31.066683 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f152d2-2364-4b95-aaf9-1a18c2c26d1d service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:31.067627 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-98902331-9ea3-402e-b2f0-00448cbd8525 None None] MFA Rules not processed for user `4099fa631da548508b1a196887a9160e`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:31.069099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-98902331-9ea3-402e-b2f0-00448cbd8525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:31.093847 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 797/1611] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:32:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:32:31.094787 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f152d2-2364-4b95-aaf9-1a18c2c26d1d service neutron] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:31.108257 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f152d2-2364-4b95-aaf9-1a18c2c26d1d service neutron] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:31.113857 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658afe3-dd0a-4274-9a5f-d2d43c7377bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:31.114711 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658afe3-dd0a-4274-9a5f-d2d43c7377bd admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:31.114993 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658afe3-dd0a-4274-9a5f-d2d43c7377bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:31.115333 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658afe3-dd0a-4274-9a5f-d2d43c7377bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:31.116021 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658afe3-dd0a-4274-9a5f-d2d43c7377bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:31.118167 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658afe3-dd0a-4274-9a5f-d2d43c7377bd admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:31.167451 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 798/1612] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:32:31 2022] POST /identity/v3/projects => generated 405 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:32:31.168839 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9def3d0-7a4a-4c56-aac6-d4da05368700 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:31.169261 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9def3d0-7a4a-4c56-aac6-d4da05368700 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:31.169547 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9def3d0-7a4a-4c56-aac6-d4da05368700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:31.193022 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-e9def3d0-7a4a-4c56-aac6-d4da05368700 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:31.195358 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-e9def3d0-7a4a-4c56-aac6-d4da05368700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:31.229837 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 815/1613] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:31.231564 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9ee87-650f-4d53-8341-0c276a31e3f8 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:31.231853 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9ee87-650f-4d53-8341-0c276a31e3f8 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:31.234412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9ee87-650f-4d53-8341-0c276a31e3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:31.246075 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:31.314897 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-73f9ee87-650f-4d53-8341-0c276a31e3f8 None None] MFA Rules not processed for user `8545e58f849249dca7f95e68295a3674`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:31.316067 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 799/1614] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:31.317049 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-73f9ee87-650f-4d53-8341-0c276a31e3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:31.317841 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6437a7-0375-44d6-8dc6-329a56804ed6 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:31.318176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6437a7-0375-44d6-8dc6-329a56804ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:31.318726 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6437a7-0375-44d6-8dc6-329a56804ed6 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:31.320005 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 800/1615] 10.222.0.49 () {54 vars in 961 bytes} [Mon Sep 26 10:32:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:32:31.320829 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6437a7-0375-44d6-8dc6-329a56804ed6 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:31.336395 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6437a7-0375-44d6-8dc6-329a56804ed6 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:31.341392 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd639e7-3b14-4042-b549-52f81d97d24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:31.342178 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd639e7-3b14-4042-b549-52f81d97d24c admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:31.342462 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd639e7-3b14-4042-b549-52f81d97d24c admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:31.342739 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd639e7-3b14-4042-b549-52f81d97d24c admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:31.343416 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd639e7-3b14-4042-b549-52f81d97d24c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:31.345355 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd639e7-3b14-4042-b549-52f81d97d24c admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:31.391538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 801/1616] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:32:31 2022] POST /identity/v3/users => generated 309 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:32:31.394241 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9e5d2-d4c6-4f39-bf7d-7a8390c0a59e None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:31.394602 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9e5d2-d4c6-4f39-bf7d-7a8390c0a59e None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:31.395103 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9e5d2-d4c6-4f39-bf7d-7a8390c0a59e None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:31.397402 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 802/1617] 10.222.0.49 () {54 vars in 961 bytes} [Mon Sep 26 10:32:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:32:31.408636 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14984e-7b42-4af3-8e06-2273853c5831 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:31.409207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14984e-7b42-4af3-8e06-2273853c5831 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:31.409555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14984e-7b42-4af3-8e06-2273853c5831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:31.416867 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 816/1618] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:31.416867 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9ee87-650f-4d53-8341-0c276a31e3f8 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:31.485780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-bc14984e-7b42-4af3-8e06-2273853c5831 None None] MFA Rules not processed for user `cee41014d5d043888c6b399710bccab4`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:31.488140 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-bc14984e-7b42-4af3-8e06-2273853c5831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:31.542976 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9ee87-650f-4d53-8341-0c276a31e3f8 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:31.556287 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81517f0f-e936-47f2-9bba-a9f719e751ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:31.557132 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81517f0f-e936-47f2-9bba-a9f719e751ef service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:31.558221 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81517f0f-e936-47f2-9bba-a9f719e751ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:31.558652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81517f0f-e936-47f2-9bba-a9f719e751ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:31.574213 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:31.583226 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81517f0f-e936-47f2-9bba-a9f719e751ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:31.584820 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81517f0f-e936-47f2-9bba-a9f719e751ef service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:31.606603 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 817/1619] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:32:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:32:31.609405 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81517f0f-e936-47f2-9bba-a9f719e751ef service neutron] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:31.623140 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 803/1620] 10.222.0.49 () {58 vars in 1058 bytes} [Mon Sep 26 10:32:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:31.624536 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada14295-3def-4ae3-a3a5-cc2bb80b6e64 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:31.624918 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada14295-3def-4ae3-a3a5-cc2bb80b6e64 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:31.625226 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada14295-3def-4ae3-a3a5-cc2bb80b6e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:31.633503 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81517f0f-e936-47f2-9bba-a9f719e751ef service neutron] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:31.638887 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a11ac8-4e66-4675-8c58-ea27a7cb484b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:31.639703 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a11ac8-4e66-4675-8c58-ea27a7cb484b admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:31.640057 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a11ac8-4e66-4675-8c58-ea27a7cb484b admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:31.640395 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a11ac8-4e66-4675-8c58-ea27a7cb484b admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:31.641107 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a11ac8-4e66-4675-8c58-ea27a7cb484b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:31.642829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a11ac8-4e66-4675-8c58-ea27a7cb484b admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:31.651909 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-ada14295-3def-4ae3-a3a5-cc2bb80b6e64 None None] MFA Rules not processed for user `cee41014d5d043888c6b399710bccab4`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:31.652439 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-ada14295-3def-4ae3-a3a5-cc2bb80b6e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:31.661798 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 818/1621] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:31 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:32:31.662761 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a11ac8-4e66-4675-8c58-ea27a7cb484b admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:31.663961 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 804/1622] 10.222.0.49 () {58 vars in 1058 bytes} [Mon Sep 26 10:32:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:31.665189 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada14295-3def-4ae3-a3a5-cc2bb80b6e64 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:31.678599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada14295-3def-4ae3-a3a5-cc2bb80b6e64 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:31.689233 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5666e32-efc5-43c8-87d6-f06fdefbc1e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:31.689872 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5666e32-efc5-43c8-87d6-f06fdefbc1e4 service nova] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:31.690493 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5666e32-efc5-43c8-87d6-f06fdefbc1e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:31.690768 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5666e32-efc5-43c8-87d6-f06fdefbc1e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:31.692565 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a11ac8-4e66-4675-8c58-ea27a7cb484b admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:31.697492 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2032f-2c62-4f0b-934d-02699f350bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:31.698827 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2032f-2c62-4f0b-934d-02699f350bae admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:31.698961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2032f-2c62-4f0b-934d-02699f350bae admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:31.699229 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2032f-2c62-4f0b-934d-02699f350bae admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:31.699983 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2032f-2c62-4f0b-934d-02699f350bae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:31.701903 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2032f-2c62-4f0b-934d-02699f350bae admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:31.763681 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 819/1623] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:32:31 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:32:31.764686 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2032f-2c62-4f0b-934d-02699f350bae admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:31.823401 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5666e32-efc5-43c8-87d6-f06fdefbc1e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:31.825050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5666e32-efc5-43c8-87d6-f06fdefbc1e4 service nova] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:31.852409 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 805/1624] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:32:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:32:31.853269 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5666e32-efc5-43c8-87d6-f06fdefbc1e4 service nova] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:31.862623 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2032f-2c62-4f0b-934d-02699f350bae admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:31.876791 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5666e32-efc5-43c8-87d6-f06fdefbc1e4 service nova] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:31.878873 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dbbbd8-51bc-443a-a68e-a58cf707e858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:31.879983 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbbbd8-51bc-443a-a68e-a58cf707e858 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:31.880906 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbbbd8-51bc-443a-a68e-a58cf707e858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:31.880906 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbbbd8-51bc-443a-a68e-a58cf707e858 admin admin] PATH_INFO: `/v3/projects/2ee2f8842bb24fca8d51eb7920c581ea/users/3864e68b224c4087afd126a2c397fcd7/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:31.904774 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dbbbd8-51bc-443a-a68e-a58cf707e858 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ee2f8842bb24fca8d51eb7920c581ea, user_id=3864e68b224c4087afd126a2c397fcd7, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:31.908002 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dbbbd8-51bc-443a-a68e-a58cf707e858 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:31.931969 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 820/1625] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:32:31 2022] PUT /identity/v3/projects/2ee2f8842bb24fca8d51eb7920c581ea/users/3864e68b224c4087afd126a2c397fcd7/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:31.932913 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dbbbd8-51bc-443a-a68e-a58cf707e858 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:31.933898 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:32.024629 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6acf261-93cf-4fa8-aaa9-16f0d04d9378 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:32.025813 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6acf261-93cf-4fa8-aaa9-16f0d04d9378 service nova] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:32.025813 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6acf261-93cf-4fa8-aaa9-16f0d04d9378 service nova] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:32.026631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6acf261-93cf-4fa8-aaa9-16f0d04d9378 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:32.034381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dbbbd8-51bc-443a-a68e-a58cf707e858 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:32.046299 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4e09b-e0ce-4daa-abfd-c9dfc1ccb0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:32.046922 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4e09b-e0ce-4daa-abfd-c9dfc1ccb0be admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:32.047199 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4e09b-e0ce-4daa-abfd-c9dfc1ccb0be admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:32.049611 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4e09b-e0ce-4daa-abfd-c9dfc1ccb0be admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:32.050313 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4e09b-e0ce-4daa-abfd-c9dfc1ccb0be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:32.055039 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4e09b-e0ce-4daa-abfd-c9dfc1ccb0be admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:32.109678 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 821/1626] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:32:31 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:32:32.110761 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4e09b-e0ce-4daa-abfd-c9dfc1ccb0be admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:32.128825 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4e09b-e0ce-4daa-abfd-c9dfc1ccb0be admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:32.139878 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee4808-c187-4ada-80f9-c5986907e139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:32.141521 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee4808-c187-4ada-80f9-c5986907e139 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:32.141882 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee4808-c187-4ada-80f9-c5986907e139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:32.142596 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee4808-c187-4ada-80f9-c5986907e139 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:32.143563 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee4808-c187-4ada-80f9-c5986907e139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:32.146468 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee4808-c187-4ada-80f9-c5986907e139 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:32.165353 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 822/1627] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:32 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:32:32.165353 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee4808-c187-4ada-80f9-c5986907e139 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:32.168325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6acf261-93cf-4fa8-aaa9-16f0d04d9378 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:32.170269 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6acf261-93cf-4fa8-aaa9-16f0d04d9378 service nova] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:32.177677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee4808-c187-4ada-80f9-c5986907e139 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:32.186319 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd7d79-1b9c-4221-91cd-df481e53e283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:32.186946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd7d79-1b9c-4221-91cd-df481e53e283 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:32.187421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd7d79-1b9c-4221-91cd-df481e53e283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:32.187800 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd7d79-1b9c-4221-91cd-df481e53e283 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:32.188629 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd7d79-1b9c-4221-91cd-df481e53e283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:32.190573 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd7d79-1b9c-4221-91cd-df481e53e283 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:32.192535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 806/1628] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:32:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:32:32.194085 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6acf261-93cf-4fa8-aaa9-16f0d04d9378 service nova] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:32.204748 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 823/1629] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:32 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:32:32.208942 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6acf261-93cf-4fa8-aaa9-16f0d04d9378 service nova] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:32.209162 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd7d79-1b9c-4221-91cd-df481e53e283 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:32.214387 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702640d-6e22-4e69-9c1d-0c46826aea85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:32.215236 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702640d-6e22-4e69-9c1d-0c46826aea85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:32.215670 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702640d-6e22-4e69-9c1d-0c46826aea85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:32.216149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702640d-6e22-4e69-9c1d-0c46826aea85 admin admin] PATH_INFO: `/v3/projects/2ee2f8842bb24fca8d51eb7920c581ea/users/3864e68b224c4087afd126a2c397fcd7/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:32.218431 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702640d-6e22-4e69-9c1d-0c46826aea85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ee2f8842bb24fca8d51eb7920c581ea, user_id=3864e68b224c4087afd126a2c397fcd7, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:32.221575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702640d-6e22-4e69-9c1d-0c46826aea85 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:32.228196 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd7d79-1b9c-4221-91cd-df481e53e283 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:32.246956 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 807/1630] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:32:32 2022] PUT /identity/v3/projects/2ee2f8842bb24fca8d51eb7920c581ea/users/3864e68b224c4087afd126a2c397fcd7/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:32.256926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993255b-9fc8-4291-8f0a-f526cc422f0f None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:32.256926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993255b-9fc8-4291-8f0a-f526cc422f0f None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:32.256926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993255b-9fc8-4291-8f0a-f526cc422f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:32.263547 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:32.310940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-4993255b-9fc8-4291-8f0a-f526cc422f0f None None] MFA Rules not processed for user `3864e68b224c4087afd126a2c397fcd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:32.313548 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-4993255b-9fc8-4291-8f0a-f526cc422f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:32.340229 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cfff0d-606b-426f-85e8-74ca98e01d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:32.340899 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cfff0d-606b-426f-85e8-74ca98e01d95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:32.341273 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cfff0d-606b-426f-85e8-74ca98e01d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:32.341546 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cfff0d-606b-426f-85e8-74ca98e01d95 admin admin] PATH_INFO: `/v3/projects/6f1f454fe7a14b518efade0b0a052afc/users/8c3f8727d5a34c918f07f24b50c0eef2/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:32.373840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cfff0d-606b-426f-85e8-74ca98e01d95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f1f454fe7a14b518efade0b0a052afc, user_id=8c3f8727d5a34c918f07f24b50c0eef2, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:32.383532 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cfff0d-606b-426f-85e8-74ca98e01d95 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:32.419112 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 824/1631] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:32:32 2022] PUT /identity/v3/projects/6f1f454fe7a14b518efade0b0a052afc/users/8c3f8727d5a34c918f07f24b50c0eef2/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:32:32.423089 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cfff0d-606b-426f-85e8-74ca98e01d95 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:32.494283 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 808/1632] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:32 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:32.498191 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1be50-6464-4070-8b48-405e21b1e6b5 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:32.499778 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1be50-6464-4070-8b48-405e21b1e6b5 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:32.500089 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1be50-6464-4070-8b48-405e21b1e6b5 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:32.504773 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 809/1633] 10.222.0.49 () {54 vars in 912 bytes} [Mon Sep 26 10:32:32 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:32:32.506985 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1be50-6464-4070-8b48-405e21b1e6b5 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:32.636934 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cfff0d-606b-426f-85e8-74ca98e01d95 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:32.647453 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:32.653767 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a48466-5876-4784-a412-2717adf3134c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:32.654162 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a48466-5876-4784-a412-2717adf3134c admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:32.654503 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a48466-5876-4784-a412-2717adf3134c admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:32.654842 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a48466-5876-4784-a412-2717adf3134c admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:32.655543 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a48466-5876-4784-a412-2717adf3134c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:32.657891 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a48466-5876-4784-a412-2717adf3134c admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:32.681212 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 825/1634] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:32 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:32:32.683311 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c1b22-4033-43c5-a550-3c005eb0eeb2 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:32.683597 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c1b22-4033-43c5-a550-3c005eb0eeb2 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:32.683874 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c1b22-4033-43c5-a550-3c005eb0eeb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:32.691055 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1be50-6464-4070-8b48-405e21b1e6b5 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:32.704354 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5690e9b-a3e8-4e29-8330-7da2eb1e5505 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:32.705096 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5690e9b-a3e8-4e29-8330-7da2eb1e5505 service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:32.705470 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5690e9b-a3e8-4e29-8330-7da2eb1e5505 service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:32.705840 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5690e9b-a3e8-4e29-8330-7da2eb1e5505 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:32.709582 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-391c1b22-4033-43c5-a550-3c005eb0eeb2 None None] MFA Rules not processed for user `cee41014d5d043888c6b399710bccab4`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:32.713064 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-391c1b22-4033-43c5-a550-3c005eb0eeb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:32.827656 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 826/1635] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Sep 26 10:32:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:32.828670 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c1b22-4033-43c5-a550-3c005eb0eeb2 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:32.847567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5690e9b-a3e8-4e29-8330-7da2eb1e5505 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:32.847884 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c1b22-4033-43c5-a550-3c005eb0eeb2 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:32.849822 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5690e9b-a3e8-4e29-8330-7da2eb1e5505 service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:32.855714 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773a9b5-b3e6-4425-a63a-eb37e6a3d3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:32.856540 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773a9b5-b3e6-4425-a63a-eb37e6a3d3a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:32.857066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773a9b5-b3e6-4425-a63a-eb37e6a3d3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:32.857416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773a9b5-b3e6-4425-a63a-eb37e6a3d3a5 admin admin] PATH_INFO: `/v3/projects/6f1f454fe7a14b518efade0b0a052afc/users/8c3f8727d5a34c918f07f24b50c0eef2/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:32.860013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773a9b5-b3e6-4425-a63a-eb37e6a3d3a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f1f454fe7a14b518efade0b0a052afc, user_id=8c3f8727d5a34c918f07f24b50c0eef2, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:32.863496 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773a9b5-b3e6-4425-a63a-eb37e6a3d3a5 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:32.875061 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 810/1636] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:32:32 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:32:32.876743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f3507-1723-496b-95c6-6634661d6437 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:32.877127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f3507-1723-496b-95c6-6634661d6437 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:32.877350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 827/1637] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:32:32 2022] PUT /identity/v3/projects/6f1f454fe7a14b518efade0b0a052afc/users/8c3f8727d5a34c918f07f24b50c0eef2/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:32.877644 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f3507-1723-496b-95c6-6634661d6437 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:32.878523 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773a9b5-b3e6-4425-a63a-eb37e6a3d3a5 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:32.879747 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 811/1638] 10.222.0.49 () {54 vars in 912 bytes} [Mon Sep 26 10:32:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:32:32.891806 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b5dc6-3479-48d7-bc8b-6ca373205608 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:32.894275 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b5dc6-3479-48d7-bc8b-6ca373205608 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:32.894814 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b5dc6-3479-48d7-bc8b-6ca373205608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:32.924503 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:32.933733 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-5d9b5dc6-3479-48d7-bc8b-6ca373205608 None None] MFA Rules not processed for user `cee41014d5d043888c6b399710bccab4`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:32.933733 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-5d9b5dc6-3479-48d7-bc8b-6ca373205608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:33.035311 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773a9b5-b3e6-4425-a63a-eb37e6a3d3a5 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:33.049550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8d99c-e478-4c45-8288-47b11cb237af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:33.050234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8d99c-e478-4c45-8288-47b11cb237af service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:33.050517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8d99c-e478-4c45-8288-47b11cb237af service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:33.050795 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8d99c-e478-4c45-8288-47b11cb237af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:33.055341 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 812/1639] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Sep 26 10:32:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:33.064099 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc41a44-5c50-4ae5-8793-d85ce7834908 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:33.064520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc41a44-5c50-4ae5-8793-d85ce7834908 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:33.064867 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc41a44-5c50-4ae5-8793-d85ce7834908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:33.097112 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8d99c-e478-4c45-8288-47b11cb237af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:33.101274 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8d99c-e478-4c45-8288-47b11cb237af service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:33.110385 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-fbc41a44-5c50-4ae5-8793-d85ce7834908 None None] MFA Rules not processed for user `8c3f8727d5a34c918f07f24b50c0eef2`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:33.112246 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-fbc41a44-5c50-4ae5-8793-d85ce7834908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:33.116894 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 828/1640] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:32:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:32:33.117927 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8d99c-e478-4c45-8288-47b11cb237af service cinder] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:33.129041 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8d99c-e478-4c45-8288-47b11cb237af service cinder] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:33.133680 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b399f95-b467-442b-b1b7-732caab27093 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:33.134428 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b399f95-b467-442b-b1b7-732caab27093 service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:33.134588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b399f95-b467-442b-b1b7-732caab27093 service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:33.134915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b399f95-b467-442b-b1b7-732caab27093 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:33.148654 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b399f95-b467-442b-b1b7-732caab27093 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:33.150114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b399f95-b467-442b-b1b7-732caab27093 service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:33.174552 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 829/1641] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:32:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:32:33.295552 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 813/1642] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:33 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:33.426191 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b399f95-b467-442b-b1b7-732caab27093 service cinder] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:33.433320 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc41a44-5c50-4ae5-8793-d85ce7834908 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:33.440141 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:33.450556 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc41a44-5c50-4ae5-8793-d85ce7834908 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:33.461964 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97fff0-2cc5-40cd-b9c6-d2e4fe5d627b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:33.462739 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97fff0-2cc5-40cd-b9c6-d2e4fe5d627b service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:33.463080 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97fff0-2cc5-40cd-b9c6-d2e4fe5d627b service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:33.463283 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97fff0-2cc5-40cd-b9c6-d2e4fe5d627b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:33.487355 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d97fff0-2cc5-40cd-b9c6-d2e4fe5d627b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:33.490424 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d97fff0-2cc5-40cd-b9c6-d2e4fe5d627b service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:33.527006 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 814/1643] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:32:33 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:32:33.620762 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b399f95-b467-442b-b1b7-732caab27093 service cinder] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:33.637771 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92919df5-364f-4021-ad71-5ccd1569d71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:33.638582 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92919df5-364f-4021-ad71-5ccd1569d71b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:33.638870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92919df5-364f-4021-ad71-5ccd1569d71b admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:33.640711 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92919df5-364f-4021-ad71-5ccd1569d71b admin admin] PATH_INFO: `/v3/users/d06fcd37160142ecaf45f9605416aace` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:33.643900 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92919df5-364f-4021-ad71-5ccd1569d71b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d06fcd37160142ecaf45f9605416aace)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:33.645153 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: WARNING py.warnings [None req-92919df5-364f-4021-ad71-5ccd1569d71b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 10:32:33.645153 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: warnings.warn(msg) Sep 26 10:32:33.645153 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: Sep 26 10:32:33.648291 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92919df5-364f-4021-ad71-5ccd1569d71b admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:33.799037 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97fff0-2cc5-40cd-b9c6-d2e4fe5d627b service cinder] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:33.805025 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97fff0-2cc5-40cd-b9c6-d2e4fe5d627b service cinder] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:33.812273 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-92919df5-364f-4021-ad71-5ccd1569d71b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd06fcd37160142ecaf45f9605416aace'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:33.813015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-92919df5-364f-4021-ad71-5ccd1569d71b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd06fcd37160142ecaf45f9605416aace'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:33.814180 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704058b0-dd8f-4816-a4f0-e68d79b12f9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:33.814999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704058b0-dd8f-4816-a4f0-e68d79b12f9e service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:33.815289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704058b0-dd8f-4816-a4f0-e68d79b12f9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:33.815564 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704058b0-dd8f-4816-a4f0-e68d79b12f9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:33.825515 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-704058b0-dd8f-4816-a4f0-e68d79b12f9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:33.827967 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-92919df5-364f-4021-ad71-5ccd1569d71b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd06fcd37160142ecaf45f9605416aace'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:33.883881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-92919df5-364f-4021-ad71-5ccd1569d71b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd06fcd37160142ecaf45f9605416aace'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:33.888380 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-92919df5-364f-4021-ad71-5ccd1569d71b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd06fcd37160142ecaf45f9605416aace'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:33.919790 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 830/1644] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:32:33 2022] DELETE /identity/v3/users/d06fcd37160142ecaf45f9605416aace => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:33.923792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92919df5-364f-4021-ad71-5ccd1569d71b admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:33.948525 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-92919df5-364f-4021-ad71-5ccd1569d71b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:34.037089 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704058b0-dd8f-4816-a4f0-e68d79b12f9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:34.037089 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704058b0-dd8f-4816-a4f0-e68d79b12f9e service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:34.037089 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 815/1645] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:32:33 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:32:34.042516 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b958449-cdb0-4316-a894-7115e4c1e84b None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:34.042516 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b958449-cdb0-4316-a894-7115e4c1e84b None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:34.042516 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b958449-cdb0-4316-a894-7115e4c1e84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:34.048749 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:34.104656 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-8b958449-cdb0-4316-a894-7115e4c1e84b None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:34.107292 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-8b958449-cdb0-4316-a894-7115e4c1e84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:34.264587 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92919df5-364f-4021-ad71-5ccd1569d71b admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:34.276630 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 816/1646] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:34.293206 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eadab2-0b34-4694-9ece-96311fe3f0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:34.294012 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eadab2-0b34-4694-9ece-96311fe3f0e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:34.294342 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eadab2-0b34-4694-9ece-96311fe3f0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:34.294621 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eadab2-0b34-4694-9ece-96311fe3f0e7 admin admin] PATH_INFO: `/v3/users/4099fa631da548508b1a196887a9160e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:34.301832 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eadab2-0b34-4694-9ece-96311fe3f0e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4099fa631da548508b1a196887a9160e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:34.303949 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eadab2-0b34-4694-9ece-96311fe3f0e7 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:34.414457 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b958449-cdb0-4316-a894-7115e4c1e84b None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:34.416312 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-8b958449-cdb0-4316-a894-7115e4c1e84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:34.429680 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-00eadab2-0b34-4694-9ece-96311fe3f0e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4099fa631da548508b1a196887a9160e'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:34.432047 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-00eadab2-0b34-4694-9ece-96311fe3f0e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4099fa631da548508b1a196887a9160e'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:34.445688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-00eadab2-0b34-4694-9ece-96311fe3f0e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4099fa631da548508b1a196887a9160e'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:34.483833 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-00eadab2-0b34-4694-9ece-96311fe3f0e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4099fa631da548508b1a196887a9160e'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:34.486447 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-00eadab2-0b34-4694-9ece-96311fe3f0e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4099fa631da548508b1a196887a9160e'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:34.523146 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 831/1647] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:32:33 2022] DELETE /identity/v3/users/4099fa631da548508b1a196887a9160e => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:34.564777 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29fef5-04e8-4ce1-9d96-4383ce18a325 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:34.564777 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29fef5-04e8-4ce1-9d96-4383ce18a325 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:34.564777 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29fef5-04e8-4ce1-9d96-4383ce18a325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:34.564777 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-9a29fef5-04e8-4ce1-9d96-4383ce18a325 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:34.564777 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-9a29fef5-04e8-4ce1-9d96-4383ce18a325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:34.609351 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b958449-cdb0-4316-a894-7115e4c1e84b None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:34.674589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:34.692744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 832/1648] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:34.695884 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff489506-6a9c-4d94-bae5-65890ed5ba45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:34.696754 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff489506-6a9c-4d94-bae5-65890ed5ba45 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:34.697196 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff489506-6a9c-4d94-bae5-65890ed5ba45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:34.697583 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff489506-6a9c-4d94-bae5-65890ed5ba45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:34.700443 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-ff489506-6a9c-4d94-bae5-65890ed5ba45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:34.705782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a29fef5-04e8-4ce1-9d96-4383ce18a325 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:34.707704 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-9a29fef5-04e8-4ce1-9d96-4383ce18a325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:34.723196 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff489506-6a9c-4d94-bae5-65890ed5ba45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:34.724654 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff489506-6a9c-4d94-bae5-65890ed5ba45 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:34.738107 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 817/1649] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:32:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:32:34.742914 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a29fef5-04e8-4ce1-9d96-4383ce18a325 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:34.754842 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbe28d-e26d-451f-80ce-610598418290 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:34.756341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbe28d-e26d-451f-80ce-610598418290 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:34.756851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbe28d-e26d-451f-80ce-610598418290 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:34.757256 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbe28d-e26d-451f-80ce-610598418290 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:34.789596 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fbe28d-e26d-451f-80ce-610598418290 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:34.794637 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fbe28d-e26d-451f-80ce-610598418290 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:34.816946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 833/1650] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:32:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:32:37.219089 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff489506-6a9c-4d94-bae5-65890ed5ba45 service neutron] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:37.220975 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-ff489506-6a9c-4d94-bae5-65890ed5ba45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:37.253912 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff489506-6a9c-4d94-bae5-65890ed5ba45 service neutron] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:37.271966 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ced43a-ce4b-4432-b90b-a9296889ad2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:37.272808 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ced43a-ce4b-4432-b90b-a9296889ad2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:37.273201 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ced43a-ce4b-4432-b90b-a9296889ad2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:37.274835 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ced43a-ce4b-4432-b90b-a9296889ad2f admin admin] PATH_INFO: `/v3/projects/8472411865af4f63b0d54d57ce2aa31a` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:37.277232 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ced43a-ce4b-4432-b90b-a9296889ad2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8472411865af4f63b0d54d57ce2aa31a)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:37.279345 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: WARNING py.warnings [None req-50ced43a-ce4b-4432-b90b-a9296889ad2f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 10:32:37.279345 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: warnings.warn(msg) Sep 26 10:32:37.279345 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: Sep 26 10:32:37.281505 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ced43a-ce4b-4432-b90b-a9296889ad2f admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:37.448591 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-50ced43a-ce4b-4432-b90b-a9296889ad2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8472411865af4f63b0d54d57ce2aa31a', 'request_id': 'req-50ced43a-ce4b-4432-b90b-a9296889ad2f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:37.480755 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-50ced43a-ce4b-4432-b90b-a9296889ad2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8472411865af4f63b0d54d57ce2aa31a', 'request_id': 'req-50ced43a-ce4b-4432-b90b-a9296889ad2f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:37.488837 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-50ced43a-ce4b-4432-b90b-a9296889ad2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8472411865af4f63b0d54d57ce2aa31a', 'request_id': 'req-50ced43a-ce4b-4432-b90b-a9296889ad2f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:37.520489 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-50ced43a-ce4b-4432-b90b-a9296889ad2f admin admin] The token cache is being invalidate because project 8472411865af4f63b0d54d57ce2aa31a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:32:37.521316 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-50ced43a-ce4b-4432-b90b-a9296889ad2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:37.523494 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-50ced43a-ce4b-4432-b90b-a9296889ad2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:37.526622 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 818/1651] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:32:37 2022] DELETE /identity/v3/projects/8472411865af4f63b0d54d57ce2aa31a => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:37.531986 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff2b69-42a1-4a52-a2ce-70e045ce84fb None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:37.532528 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff2b69-42a1-4a52-a2ce-70e045ce84fb None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:37.533507 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff2b69-42a1-4a52-a2ce-70e045ce84fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:37.595798 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-caff2b69-42a1-4a52-a2ce-70e045ce84fb None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:37.600603 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-caff2b69-42a1-4a52-a2ce-70e045ce84fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:37.650858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:37.721481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 834/1652] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:37.729849 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2a0d6-da3d-4730-a394-f3e70a6aa89e None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:37.730056 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2a0d6-da3d-4730-a394-f3e70a6aa89e None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:37.730359 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2a0d6-da3d-4730-a394-f3e70a6aa89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:37.772793 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-bfe2a0d6-da3d-4730-a394-f3e70a6aa89e None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:37.774284 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-bfe2a0d6-da3d-4730-a394-f3e70a6aa89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:37.790794 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 819/1653] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:37.796291 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff2b69-42a1-4a52-a2ce-70e045ce84fb None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:37.814099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff2b69-42a1-4a52-a2ce-70e045ce84fb None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:37.820767 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e2c3d-98af-42c9-bb3d-ba388d496760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:37.821394 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e2c3d-98af-42c9-bb3d-ba388d496760 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:37.822128 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e2c3d-98af-42c9-bb3d-ba388d496760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:37.822409 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e2c3d-98af-42c9-bb3d-ba388d496760 admin admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:37.823596 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e2c3d-98af-42c9-bb3d-ba388d496760 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:32:37.823596 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e2c3d-98af-42c9-bb3d-ba388d496760 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:37.825016 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e2c3d-98af-42c9-bb3d-ba388d496760 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:37.842158 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 835/1654] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:32:37.843335 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2a0d6-da3d-4730-a394-f3e70a6aa89e None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:37.860127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2a0d6-da3d-4730-a394-f3e70a6aa89e None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:37.874888 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8939c4-1dbe-436f-a37d-92b0381bd504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:37.875806 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8939c4-1dbe-436f-a37d-92b0381bd504 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:37.876874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8939c4-1dbe-436f-a37d-92b0381bd504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:37.877959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8939c4-1dbe-436f-a37d-92b0381bd504 admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:37.881074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8939c4-1dbe-436f-a37d-92b0381bd504 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:37.883121 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8939c4-1dbe-436f-a37d-92b0381bd504 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:37.953207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 820/1655] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:32:37 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:32:37.954402 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e2c3d-98af-42c9-bb3d-ba388d496760 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:37.970729 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e2c3d-98af-42c9-bb3d-ba388d496760 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:38.164336 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00989d4b-e742-4358-b30b-d76c02437428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:38.165733 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00989d4b-e742-4358-b30b-d76c02437428 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:38.168231 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00989d4b-e742-4358-b30b-d76c02437428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:38.168886 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00989d4b-e742-4358-b30b-d76c02437428 admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:38.169757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00989d4b-e742-4358-b30b-d76c02437428 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:38.174735 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00989d4b-e742-4358-b30b-d76c02437428 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:38.281653 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 836/1656] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:32:37 2022] POST /identity/v3/users => generated 287 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:32:38.300141 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8939c4-1dbe-436f-a37d-92b0381bd504 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:38.322485 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8939c4-1dbe-436f-a37d-92b0381bd504 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:38.340476 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c02309-962f-429b-bb16-3aeab4000d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:38.340476 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c02309-962f-429b-bb16-3aeab4000d83 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:38.340476 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c02309-962f-429b-bb16-3aeab4000d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:38.340767 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c02309-962f-429b-bb16-3aeab4000d83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:38.341455 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c02309-962f-429b-bb16-3aeab4000d83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:38.347035 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c02309-962f-429b-bb16-3aeab4000d83 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:38.376701 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 821/1657] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:32:38.380824 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00989d4b-e742-4358-b30b-d76c02437428 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:38.397991 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00989d4b-e742-4358-b30b-d76c02437428 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:38.411049 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2973c0-f42e-489b-8d4a-44080ef78aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:38.411991 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2973c0-f42e-489b-8d4a-44080ef78aaa admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:38.412748 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2973c0-f42e-489b-8d4a-44080ef78aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:38.413121 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2973c0-f42e-489b-8d4a-44080ef78aaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:38.414103 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2973c0-f42e-489b-8d4a-44080ef78aaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:38.416176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2973c0-f42e-489b-8d4a-44080ef78aaa admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:38.455659 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 837/1658] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:32:38.461821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c02309-962f-429b-bb16-3aeab4000d83 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:38.495982 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c02309-962f-429b-bb16-3aeab4000d83 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:38.503085 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f66e5-c72d-4940-890a-72d2d3ca5199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:38.504114 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f66e5-c72d-4940-890a-72d2d3ca5199 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:38.504114 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f66e5-c72d-4940-890a-72d2d3ca5199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:38.504306 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f66e5-c72d-4940-890a-72d2d3ca5199 admin admin] PATH_INFO: `/v3/projects/08019dd7d2454aceb518d7db58ee3278/users/39ead3dfe9cf4788b66309ecefec6ca8/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:38.534237 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f66e5-c72d-4940-890a-72d2d3ca5199 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08019dd7d2454aceb518d7db58ee3278, user_id=39ead3dfe9cf4788b66309ecefec6ca8, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:38.539603 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f66e5-c72d-4940-890a-72d2d3ca5199 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:38.570578 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 822/1659] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:32:38 2022] PUT /identity/v3/projects/08019dd7d2454aceb518d7db58ee3278/users/39ead3dfe9cf4788b66309ecefec6ca8/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:38.582771 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4889b9a-9b05-47f9-962c-1aaa5a98be8c None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:38.583256 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4889b9a-9b05-47f9-962c-1aaa5a98be8c None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:38.583416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4889b9a-9b05-47f9-962c-1aaa5a98be8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:38.647258 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-a4889b9a-9b05-47f9-962c-1aaa5a98be8c None None] MFA Rules not processed for user `39ead3dfe9cf4788b66309ecefec6ca8`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:38.648660 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-a4889b9a-9b05-47f9-962c-1aaa5a98be8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:38.659943 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1805b15-0d2e-4d5c-8d5d-5625507be499 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:38.660236 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1805b15-0d2e-4d5c-8d5d-5625507be499 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:38.660541 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1805b15-0d2e-4d5c-8d5d-5625507be499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:38.721207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-c1805b15-0d2e-4d5c-8d5d-5625507be499 None None] MFA Rules not processed for user `398f3e872ad9499fbb26cc48ca9544d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:38.723013 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-c1805b15-0d2e-4d5c-8d5d-5625507be499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:38.770517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 838/1660] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:38.770517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4889b9a-9b05-47f9-962c-1aaa5a98be8c None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:38.770517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-a4889b9a-9b05-47f9-962c-1aaa5a98be8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:38.819153 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:38.846932 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:38.862685 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 823/1661] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:32:38.864009 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1805b15-0d2e-4d5c-8d5d-5625507be499 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:38.935624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4889b9a-9b05-47f9-962c-1aaa5a98be8c None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:38.941877 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937818e-76be-461a-9364-e541fe740d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:38.942583 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937818e-76be-461a-9364-e541fe740d55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:38.942858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937818e-76be-461a-9364-e541fe740d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:38.943313 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937818e-76be-461a-9364-e541fe740d55 admin admin] PATH_INFO: `/v3/projects/34ab3676c5a24990a49c71f0e5055105` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:38.944973 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937818e-76be-461a-9364-e541fe740d55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34ab3676c5a24990a49c71f0e5055105)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:38.947216 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937818e-76be-461a-9364-e541fe740d55 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:39.029387 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1805b15-0d2e-4d5c-8d5d-5625507be499 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:39.036269 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b0d6b-b297-4249-b497-c09dc6fe35b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:39.036768 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b0d6b-b297-4249-b497-c09dc6fe35b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:39.037045 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b0d6b-b297-4249-b497-c09dc6fe35b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:39.037318 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b0d6b-b297-4249-b497-c09dc6fe35b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:39.038124 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b0d6b-b297-4249-b497-c09dc6fe35b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:39.040323 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b0d6b-b297-4249-b497-c09dc6fe35b9 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:39.088548 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-9937818e-76be-461a-9364-e541fe740d55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34ab3676c5a24990a49c71f0e5055105', 'request_id': 'req-9937818e-76be-461a-9364-e541fe740d55'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:39.102512 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-9937818e-76be-461a-9364-e541fe740d55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34ab3676c5a24990a49c71f0e5055105', 'request_id': 'req-9937818e-76be-461a-9364-e541fe740d55'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:39.113055 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 824/1662] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:32:38 2022] POST /identity/v3/projects => generated 373 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:32:39.115241 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca23836-7d41-4ddf-99d8-7bd52278d45b None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:39.115958 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca23836-7d41-4ddf-99d8-7bd52278d45b None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:39.116629 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca23836-7d41-4ddf-99d8-7bd52278d45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:39.122554 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-9937818e-76be-461a-9364-e541fe740d55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34ab3676c5a24990a49c71f0e5055105', 'request_id': 'req-9937818e-76be-461a-9364-e541fe740d55'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:39.142761 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-9937818e-76be-461a-9364-e541fe740d55 admin admin] The token cache is being invalidate because project 34ab3676c5a24990a49c71f0e5055105 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:32:39.143064 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-9937818e-76be-461a-9364-e541fe740d55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:39.146547 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-9937818e-76be-461a-9364-e541fe740d55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:39.152380 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 839/1663] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:32:38 2022] DELETE /identity/v3/projects/34ab3676c5a24990a49c71f0e5055105 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:39.154436 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937818e-76be-461a-9364-e541fe740d55 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:39.162582 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-9937818e-76be-461a-9364-e541fe740d55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:39.170112 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-9ca23836-7d41-4ddf-99d8-7bd52278d45b None None] MFA Rules not processed for user `398f3e872ad9499fbb26cc48ca9544d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:39.171678 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-9ca23836-7d41-4ddf-99d8-7bd52278d45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:39.292586 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 825/1664] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:39.293566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca23836-7d41-4ddf-99d8-7bd52278d45b None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:39.295191 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-9ca23836-7d41-4ddf-99d8-7bd52278d45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:39.362840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937818e-76be-461a-9364-e541fe740d55 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:39.475649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca23836-7d41-4ddf-99d8-7bd52278d45b None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:39.481451 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a7835-9a42-4d57-9d09-df3e54b04a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:39.483137 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a7835-9a42-4d57-9d09-df3e54b04a3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:39.483137 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a7835-9a42-4d57-9d09-df3e54b04a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:39.483137 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a7835-9a42-4d57-9d09-df3e54b04a3a admin admin] PATH_INFO: `/v3/users/3864e68b224c4087afd126a2c397fcd7` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:39.484804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06a7835-9a42-4d57-9d09-df3e54b04a3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3864e68b224c4087afd126a2c397fcd7)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:39.487335 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06a7835-9a42-4d57-9d09-df3e54b04a3a admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:39.546534 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db0133-f64f-4373-a964-ae35200c71a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:39.547473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db0133-f64f-4373-a964-ae35200c71a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:39.548448 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db0133-f64f-4373-a964-ae35200c71a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:39.548547 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db0133-f64f-4373-a964-ae35200c71a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:39.553209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db0133-f64f-4373-a964-ae35200c71a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:39.556496 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db0133-f64f-4373-a964-ae35200c71a5 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:39.580213 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:39.611968 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 840/1665] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:32:39 2022] POST /identity/v3/users => generated 293 bytes in 459 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:32:39.613536 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db0133-f64f-4373-a964-ae35200c71a5 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:39.614523 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:39.617355 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-e7db0133-f64f-4373-a964-ae35200c71a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:39.708636 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-f06a7835-9a42-4d57-9d09-df3e54b04a3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3864e68b224c4087afd126a2c397fcd7'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:39.711544 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-f06a7835-9a42-4d57-9d09-df3e54b04a3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3864e68b224c4087afd126a2c397fcd7'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:39.715622 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-f06a7835-9a42-4d57-9d09-df3e54b04a3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3864e68b224c4087afd126a2c397fcd7'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:39.738426 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-f06a7835-9a42-4d57-9d09-df3e54b04a3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3864e68b224c4087afd126a2c397fcd7'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:39.785005 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-f06a7835-9a42-4d57-9d09-df3e54b04a3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3864e68b224c4087afd126a2c397fcd7'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:39.818362 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 826/1666] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:32:39 2022] DELETE /identity/v3/users/3864e68b224c4087afd126a2c397fcd7 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:39.819262 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a7835-9a42-4d57-9d09-df3e54b04a3a admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:39.822334 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-f06a7835-9a42-4d57-9d09-df3e54b04a3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:39.851907 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db0133-f64f-4373-a964-ae35200c71a5 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:40.027313 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a608325-ec3b-4575-9116-d31196d64a27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:40.028156 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a608325-ec3b-4575-9116-d31196d64a27 service nova] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:40.028650 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a608325-ec3b-4575-9116-d31196d64a27 service nova] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:40.028929 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a608325-ec3b-4575-9116-d31196d64a27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:40.037216 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-7a608325-ec3b-4575-9116-d31196d64a27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:40.098081 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a7835-9a42-4d57-9d09-df3e54b04a3a admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:40.109589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315e3a2-b185-458c-98a8-2b8572845dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:40.110340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315e3a2-b185-458c-98a8-2b8572845dbf admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:40.114429 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315e3a2-b185-458c-98a8-2b8572845dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:40.114857 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315e3a2-b185-458c-98a8-2b8572845dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:40.115662 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c315e3a2-b185-458c-98a8-2b8572845dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:40.117686 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c315e3a2-b185-458c-98a8-2b8572845dbf admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:40.141434 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 827/1667] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:32:40.144031 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315e3a2-b185-458c-98a8-2b8572845dbf admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:40.164971 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315e3a2-b185-458c-98a8-2b8572845dbf admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:40.175407 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4a6b2-c9a2-4028-a919-bce78bf1578d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:40.176243 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4a6b2-c9a2-4028-a919-bce78bf1578d admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:40.177884 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4a6b2-c9a2-4028-a919-bce78bf1578d admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:40.178337 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4a6b2-c9a2-4028-a919-bce78bf1578d admin admin] PATH_INFO: `/v3/projects/934748f0880f4d4e9c6eea10fdb2f4a8/users/236eb289b08544a0ae17ebd5c353b4a7/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:40.194130 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a608325-ec3b-4575-9116-d31196d64a27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:40.196652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a608325-ec3b-4575-9116-d31196d64a27 service nova] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:40.206902 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4a6b2-c9a2-4028-a919-bce78bf1578d admin admin] RBAC: Authorizing `identity:create_grant(project_id=934748f0880f4d4e9c6eea10fdb2f4a8, user_id=236eb289b08544a0ae17ebd5c353b4a7, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:40.209889 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4a6b2-c9a2-4028-a919-bce78bf1578d admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:40.224125 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 841/1668] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:32:39 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:32:40.233325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 828/1669] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:32:40 2022] PUT /identity/v3/projects/934748f0880f4d4e9c6eea10fdb2f4a8/users/236eb289b08544a0ae17ebd5c353b4a7/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:40.237297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a608325-ec3b-4575-9116-d31196d64a27 service nova] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:40.256695 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a608325-ec3b-4575-9116-d31196d64a27 service nova] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:40.299750 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:40.418185 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c70b6-3d89-491e-9c41-b540e56debb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:40.419541 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c70b6-3d89-491e-9c41-b540e56debb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:40.419957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c70b6-3d89-491e-9c41-b540e56debb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:40.420363 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c70b6-3d89-491e-9c41-b540e56debb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:40.421169 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c70b6-3d89-491e-9c41-b540e56debb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:40.424059 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c70b6-3d89-491e-9c41-b540e56debb0 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:40.454589 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 842/1670] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:32:40.458679 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4a6b2-c9a2-4028-a919-bce78bf1578d admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:40.476494 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4a6b2-c9a2-4028-a919-bce78bf1578d admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:40.485161 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495627b-90b9-490f-8aef-f1436a5351e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:40.486268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495627b-90b9-490f-8aef-f1436a5351e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:40.486758 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495627b-90b9-490f-8aef-f1436a5351e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:40.487221 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495627b-90b9-490f-8aef-f1436a5351e3 admin admin] PATH_INFO: `/v3/projects/934748f0880f4d4e9c6eea10fdb2f4a8/users/236eb289b08544a0ae17ebd5c353b4a7/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:40.490006 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495627b-90b9-490f-8aef-f1436a5351e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=934748f0880f4d4e9c6eea10fdb2f4a8, user_id=236eb289b08544a0ae17ebd5c353b4a7, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:40.492883 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495627b-90b9-490f-8aef-f1436a5351e3 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:40.532424 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 829/1671] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:32:40 2022] PUT /identity/v3/projects/934748f0880f4d4e9c6eea10fdb2f4a8/users/236eb289b08544a0ae17ebd5c353b4a7/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:32:40.534325 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31b175-9c52-41be-b46a-ed23283048fb None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:40.534615 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31b175-9c52-41be-b46a-ed23283048fb None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:40.534889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31b175-9c52-41be-b46a-ed23283048fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:40.612082 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-7a31b175-9c52-41be-b46a-ed23283048fb None None] MFA Rules not processed for user `236eb289b08544a0ae17ebd5c353b4a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:40.613423 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-7a31b175-9c52-41be-b46a-ed23283048fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:40.809086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 843/1672] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:40.822420 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495627b-90b9-490f-8aef-f1436a5351e3 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:40.824433 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-4495627b-90b9-490f-8aef-f1436a5351e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:40.977815 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495627b-90b9-490f-8aef-f1436a5351e3 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:40.998335 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a478e1-fa9c-4210-af73-c58c86f0a27a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:40.999758 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a478e1-fa9c-4210-af73-c58c86f0a27a service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:41.000058 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a478e1-fa9c-4210-af73-c58c86f0a27a service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:41.002257 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a478e1-fa9c-4210-af73-c58c86f0a27a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:41.040703 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a478e1-fa9c-4210-af73-c58c86f0a27a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:41.044652 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a478e1-fa9c-4210-af73-c58c86f0a27a service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:41.093265 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31b175-9c52-41be-b46a-ed23283048fb None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:41.094845 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 830/1673] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:32:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:32:41.095179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-7a31b175-9c52-41be-b46a-ed23283048fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:41.315517 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a478e1-fa9c-4210-af73-c58c86f0a27a service cinder] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:41.317409 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-24a478e1-fa9c-4210-af73-c58c86f0a27a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:41.321550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31b175-9c52-41be-b46a-ed23283048fb None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:41.341255 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f08505-e881-41b4-a9df-0a40370efb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:41.342141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f08505-e881-41b4-a9df-0a40370efb40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:41.342423 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f08505-e881-41b4-a9df-0a40370efb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:41.342696 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f08505-e881-41b4-a9df-0a40370efb40 admin admin] PATH_INFO: `/v3/projects/2ee2f8842bb24fca8d51eb7920c581ea` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:41.344307 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a478e1-fa9c-4210-af73-c58c86f0a27a service cinder] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:41.346847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f08505-e881-41b4-a9df-0a40370efb40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ee2f8842bb24fca8d51eb7920c581ea)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:41.349069 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f08505-e881-41b4-a9df-0a40370efb40 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:41.359757 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69decd82-3604-4200-b740-da8d6f7b12c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:41.369319 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69decd82-3604-4200-b740-da8d6f7b12c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:41.373798 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69decd82-3604-4200-b740-da8d6f7b12c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:41.374666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69decd82-3604-4200-b740-da8d6f7b12c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:41.382090 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-69decd82-3604-4200-b740-da8d6f7b12c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:41.388078 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:41.462550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:41.465149 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-e6f08505-e881-41b4-a9df-0a40370efb40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ee2f8842bb24fca8d51eb7920c581ea', 'request_id': 'req-e6f08505-e881-41b4-a9df-0a40370efb40'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:41.475139 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-e6f08505-e881-41b4-a9df-0a40370efb40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ee2f8842bb24fca8d51eb7920c581ea', 'request_id': 'req-e6f08505-e881-41b4-a9df-0a40370efb40'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:41.490973 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-e6f08505-e881-41b4-a9df-0a40370efb40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ee2f8842bb24fca8d51eb7920c581ea', 'request_id': 'req-e6f08505-e881-41b4-a9df-0a40370efb40'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:41.506259 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-e6f08505-e881-41b4-a9df-0a40370efb40 admin admin] The token cache is being invalidate because project 2ee2f8842bb24fca8d51eb7920c581ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:32:41.506856 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-e6f08505-e881-41b4-a9df-0a40370efb40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:41.508007 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-e6f08505-e881-41b4-a9df-0a40370efb40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:41.510597 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 844/1674] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:32:41 2022] DELETE /identity/v3/projects/2ee2f8842bb24fca8d51eb7920c581ea => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:41.513523 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717c906-6a9f-4678-977d-481f1d36d2ec None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:41.514144 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717c906-6a9f-4678-977d-481f1d36d2ec None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:41.514808 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717c906-6a9f-4678-977d-481f1d36d2ec None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:41.517043 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 845/1675] 10.222.0.49 () {54 vars in 912 bytes} [Mon Sep 26 10:32:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:32:41.518674 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d135dab-dcd4-418a-a11e-a4dbc62c4103 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:41.519093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d135dab-dcd4-418a-a11e-a4dbc62c4103 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:41.519518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d135dab-dcd4-418a-a11e-a4dbc62c4103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:41.542705 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69decd82-3604-4200-b740-da8d6f7b12c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:41.544391 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69decd82-3604-4200-b740-da8d6f7b12c2 service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:41.551876 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-5d135dab-dcd4-418a-a11e-a4dbc62c4103 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:41.553937 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-5d135dab-dcd4-418a-a11e-a4dbc62c4103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:41.576497 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 831/1676] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:32:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:32:41.578118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33dbba-d175-43d0-bf6f-904181308a72 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:41.578428 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33dbba-d175-43d0-bf6f-904181308a72 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:41.578771 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33dbba-d175-43d0-bf6f-904181308a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:41.637391 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-6b33dbba-d175-43d0-bf6f-904181308a72 None None] MFA Rules not processed for user `f841bf31c43b4333a3d834d38b5197e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:41.641762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-6b33dbba-d175-43d0-bf6f-904181308a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:41.662553 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 846/1677] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:41.664911 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b96c1f-da35-45cf-92a2-69eed2001c83 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:41.665380 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b96c1f-da35-45cf-92a2-69eed2001c83 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:41.665942 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b96c1f-da35-45cf-92a2-69eed2001c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:41.712878 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-e0b96c1f-da35-45cf-92a2-69eed2001c83 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:41.714959 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-e0b96c1f-da35-45cf-92a2-69eed2001c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:41.744493 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 847/1678] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:41.746710 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 832/1679] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Sep 26 10:32:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:41.747624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b96c1f-da35-45cf-92a2-69eed2001c83 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:41.758406 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33dbba-d175-43d0-bf6f-904181308a72 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:41.760176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-6b33dbba-d175-43d0-bf6f-904181308a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:41.772900 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b96c1f-da35-45cf-92a2-69eed2001c83 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:41.781392 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33dbba-d175-43d0-bf6f-904181308a72 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:41.789542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f30191-f716-45cb-9797-69e8ede8e91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:41.790861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f30191-f716-45cb-9797-69e8ede8e91f admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:41.792130 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f30191-f716-45cb-9797-69e8ede8e91f admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:41.792807 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f30191-f716-45cb-9797-69e8ede8e91f admin admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:41.800661 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f30191-f716-45cb-9797-69e8ede8e91f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:32:41.801670 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd39e8-00b3-456d-af0c-e2985fe7cb5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:41.802322 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd39e8-00b3-456d-af0c-e2985fe7cb5c service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:41.802593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd39e8-00b3-456d-af0c-e2985fe7cb5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:41.802858 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd39e8-00b3-456d-af0c-e2985fe7cb5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:41.803524 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f30191-f716-45cb-9797-69e8ede8e91f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:41.805147 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f30191-f716-45cb-9797-69e8ede8e91f admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:41.823876 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abd39e8-00b3-456d-af0c-e2985fe7cb5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:41.826289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abd39e8-00b3-456d-af0c-e2985fe7cb5c service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:41.835868 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 848/1680] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:32:41.836898 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f30191-f716-45cb-9797-69e8ede8e91f admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:41.848853 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 833/1681] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:32:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:32:41.853951 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f30191-f716-45cb-9797-69e8ede8e91f admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:41.859013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580c8b7-677e-41c1-9433-2a4d694b5b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:41.859716 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580c8b7-677e-41c1-9433-2a4d694b5b6a admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:41.860065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580c8b7-677e-41c1-9433-2a4d694b5b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:41.860554 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580c8b7-677e-41c1-9433-2a4d694b5b6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:41.861247 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4580c8b7-677e-41c1-9433-2a4d694b5b6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:41.863301 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4580c8b7-677e-41c1-9433-2a4d694b5b6a admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:41.925822 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 849/1682] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:32:41 2022] POST /identity/v3/projects => generated 365 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:32:41.926451 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd39e8-00b3-456d-af0c-e2985fe7cb5c service neutron] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:41.946540 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd39e8-00b3-456d-af0c-e2985fe7cb5c service neutron] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:42.091353 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6133d7f-547f-4991-b3ea-fed2df71c45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:42.092790 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6133d7f-547f-4991-b3ea-fed2df71c45a admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:42.093248 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6133d7f-547f-4991-b3ea-fed2df71c45a admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:42.093692 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6133d7f-547f-4991-b3ea-fed2df71c45a admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:42.094528 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6133d7f-547f-4991-b3ea-fed2df71c45a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:42.097006 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6133d7f-547f-4991-b3ea-fed2df71c45a admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:42.151896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 834/1683] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:32:41 2022] POST /identity/v3/users => generated 283 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:32:42.154066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580c8b7-677e-41c1-9433-2a4d694b5b6a admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:42.169350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580c8b7-677e-41c1-9433-2a4d694b5b6a admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:42.179138 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d91d5e-762b-4d5e-bbc8-52d34d2ddff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:42.179974 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d91d5e-762b-4d5e-bbc8-52d34d2ddff1 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:42.180481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d91d5e-762b-4d5e-bbc8-52d34d2ddff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:42.180934 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d91d5e-762b-4d5e-bbc8-52d34d2ddff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:42.181789 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d91d5e-762b-4d5e-bbc8-52d34d2ddff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:42.184943 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d91d5e-762b-4d5e-bbc8-52d34d2ddff1 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:42.209257 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6133d7f-547f-4991-b3ea-fed2df71c45a admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:42.210250 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 850/1684] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:32:42.226951 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6133d7f-547f-4991-b3ea-fed2df71c45a admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:42.238706 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c1b9e-8762-42d0-a6c8-fd2f593e66c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:42.239242 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c1b9e-8762-42d0-a6c8-fd2f593e66c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:42.239676 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c1b9e-8762-42d0-a6c8-fd2f593e66c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:42.239864 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c1b9e-8762-42d0-a6c8-fd2f593e66c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:42.240424 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c1b9e-8762-42d0-a6c8-fd2f593e66c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:42.242109 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c1b9e-8762-42d0-a6c8-fd2f593e66c7 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:42.265394 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 835/1685] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:32:42.270338 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d91d5e-762b-4d5e-bbc8-52d34d2ddff1 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:42.286905 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d91d5e-762b-4d5e-bbc8-52d34d2ddff1 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:42.306734 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8178f07-c64a-4f03-9bc0-2f4ce49abe04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:42.307926 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8178f07-c64a-4f03-9bc0-2f4ce49abe04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:42.308476 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8178f07-c64a-4f03-9bc0-2f4ce49abe04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:42.308476 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8178f07-c64a-4f03-9bc0-2f4ce49abe04 admin admin] PATH_INFO: `/v3/projects/b3e67f6c643d41349f31ff301d6c9805/users/5ac2da43f9c04812ab96e412dcf788c8/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:42.353107 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8178f07-c64a-4f03-9bc0-2f4ce49abe04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3e67f6c643d41349f31ff301d6c9805, user_id=5ac2da43f9c04812ab96e412dcf788c8, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:42.364474 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8178f07-c64a-4f03-9bc0-2f4ce49abe04 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:42.396817 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 851/1686] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:32:42 2022] PUT /identity/v3/projects/b3e67f6c643d41349f31ff301d6c9805/users/5ac2da43f9c04812ab96e412dcf788c8/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:42.408153 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3638e1-022c-4e26-8ed8-3fc6265cacff None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:42.409284 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3638e1-022c-4e26-8ed8-3fc6265cacff None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:42.409284 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3638e1-022c-4e26-8ed8-3fc6265cacff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:42.427233 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:42.487339 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-9f3638e1-022c-4e26-8ed8-3fc6265cacff None None] MFA Rules not processed for user `5ac2da43f9c04812ab96e412dcf788c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:42.487339 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-9f3638e1-022c-4e26-8ed8-3fc6265cacff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:42.545519 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8178f07-c64a-4f03-9bc0-2f4ce49abe04 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:42.547594 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-c8178f07-c64a-4f03-9bc0-2f4ce49abe04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:42.664143 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 836/1687] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:42.669606 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3638e1-022c-4e26-8ed8-3fc6265cacff None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:42.731394 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:42.745243 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8178f07-c64a-4f03-9bc0-2f4ce49abe04 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:42.759091 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b605e7-f872-4f9f-b04b-99e80a9e8807 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84b02e7351914cabb67462454b328d05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:42.759832 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b605e7-f872-4f9f-b04b-99e80a9e8807 service placement] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:42.760182 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b605e7-f872-4f9f-b04b-99e80a9e8807 service placement] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:42.762867 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b605e7-f872-4f9f-b04b-99e80a9e8807 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:42.766578 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-57b605e7-f872-4f9f-b04b-99e80a9e8807 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:42.804741 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3638e1-022c-4e26-8ed8-3fc6265cacff None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:42.808880 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b605e7-f872-4f9f-b04b-99e80a9e8807 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:42.810260 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99288b-93c0-4f5f-9701-a2216b144b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:42.811061 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99288b-93c0-4f5f-9701-a2216b144b46 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:42.811515 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99288b-93c0-4f5f-9701-a2216b144b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:42.811782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b605e7-f872-4f9f-b04b-99e80a9e8807 service placement] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:42.811880 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99288b-93c0-4f5f-9701-a2216b144b46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:42.812424 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de99288b-93c0-4f5f-9701-a2216b144b46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:42.814356 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de99288b-93c0-4f5f-9701-a2216b144b46 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:42.835583 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 852/1688] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:32:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:32:42.861179 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 837/1689] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:32:42 2022] POST /identity/v3/projects => generated 363 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:32:42.865238 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b605e7-f872-4f9f-b04b-99e80a9e8807 service placement] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:42.882907 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b605e7-f872-4f9f-b04b-99e80a9e8807 service placement] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:43.026751 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa866cf-02d0-423c-9fbd-8efe5583b409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:43.030416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa866cf-02d0-423c-9fbd-8efe5583b409 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:43.030701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa866cf-02d0-423c-9fbd-8efe5583b409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:43.030983 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa866cf-02d0-423c-9fbd-8efe5583b409 admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:43.034510 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa866cf-02d0-423c-9fbd-8efe5583b409 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:43.038924 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa866cf-02d0-423c-9fbd-8efe5583b409 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:43.079255 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 853/1690] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:32:42 2022] POST /identity/v3/users => generated 288 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:32:43.085309 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99288b-93c0-4f5f-9701-a2216b144b46 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:43.101684 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99288b-93c0-4f5f-9701-a2216b144b46 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:43.118271 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb08d62-ed4c-41e4-8f57-b9d67e746a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:43.118926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb08d62-ed4c-41e4-8f57-b9d67e746a00 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:43.119515 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb08d62-ed4c-41e4-8f57-b9d67e746a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:43.119836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb08d62-ed4c-41e4-8f57-b9d67e746a00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:43.120594 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb08d62-ed4c-41e4-8f57-b9d67e746a00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:43.126754 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb08d62-ed4c-41e4-8f57-b9d67e746a00 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:43.178989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 838/1691] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:32:43.184269 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa866cf-02d0-423c-9fbd-8efe5583b409 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:43.211962 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa866cf-02d0-423c-9fbd-8efe5583b409 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:43.225989 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af1b8a-9b1f-4d00-b9ca-1a8a103933d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:43.226782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af1b8a-9b1f-4d00-b9ca-1a8a103933d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:43.227130 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af1b8a-9b1f-4d00-b9ca-1a8a103933d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:43.227787 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af1b8a-9b1f-4d00-b9ca-1a8a103933d7 admin admin] PATH_INFO: `/v3/projects/e899bfdb17624901b92860a5150e1454/users/29f86fb4a4364e6bb4c1e36a119741ff/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:43.249264 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af1b8a-9b1f-4d00-b9ca-1a8a103933d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e899bfdb17624901b92860a5150e1454, user_id=29f86fb4a4364e6bb4c1e36a119741ff, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:43.253364 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af1b8a-9b1f-4d00-b9ca-1a8a103933d7 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:43.289729 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 854/1692] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:32:43 2022] PUT /identity/v3/projects/e899bfdb17624901b92860a5150e1454/users/29f86fb4a4364e6bb4c1e36a119741ff/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:43.291960 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb08d62-ed4c-41e4-8f57-b9d67e746a00 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:43.314170 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb08d62-ed4c-41e4-8f57-b9d67e746a00 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:43.540890 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd16b777-0a66-44ae-b2f8-832424c60882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:43.542098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16b777-0a66-44ae-b2f8-832424c60882 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:43.543906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16b777-0a66-44ae-b2f8-832424c60882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:43.547348 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16b777-0a66-44ae-b2f8-832424c60882 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:43.548582 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd16b777-0a66-44ae-b2f8-832424c60882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:43.552016 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd16b777-0a66-44ae-b2f8-832424c60882 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:43.581663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 839/1693] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:32:43.590853 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af1b8a-9b1f-4d00-b9ca-1a8a103933d7 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:43.616653 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af1b8a-9b1f-4d00-b9ca-1a8a103933d7 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:43.639439 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af845fb-355c-4c26-a34c-358f1961dce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:43.640313 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af845fb-355c-4c26-a34c-358f1961dce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:43.640602 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af845fb-355c-4c26-a34c-358f1961dce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:43.640877 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af845fb-355c-4c26-a34c-358f1961dce7 admin admin] PATH_INFO: `/v3/projects/e899bfdb17624901b92860a5150e1454/users/29f86fb4a4364e6bb4c1e36a119741ff/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:43.643056 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af845fb-355c-4c26-a34c-358f1961dce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e899bfdb17624901b92860a5150e1454, user_id=29f86fb4a4364e6bb4c1e36a119741ff, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:43.645758 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af845fb-355c-4c26-a34c-358f1961dce7 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:43.720732 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 855/1694] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:32:43 2022] PUT /identity/v3/projects/e899bfdb17624901b92860a5150e1454/users/29f86fb4a4364e6bb4c1e36a119741ff/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:43.722923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322d355-ab35-4cd0-90e5-5dec5fa43637 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:43.723372 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322d355-ab35-4cd0-90e5-5dec5fa43637 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:43.723701 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322d355-ab35-4cd0-90e5-5dec5fa43637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:43.766065 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:43.832828 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-2322d355-ab35-4cd0-90e5-5dec5fa43637 None None] MFA Rules not processed for user `29f86fb4a4364e6bb4c1e36a119741ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:43.838236 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-2322d355-ab35-4cd0-90e5-5dec5fa43637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:44.084019 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 840/1695] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:43 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:44.099350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af845fb-355c-4c26-a34c-358f1961dce7 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:44.102401 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-2af845fb-355c-4c26-a34c-358f1961dce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:44.223732 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af845fb-355c-4c26-a34c-358f1961dce7 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:44.237414 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f0bfa-438b-4f36-82d9-3677c8ca993c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:44.238562 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f0bfa-438b-4f36-82d9-3677c8ca993c service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:44.238672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f0bfa-438b-4f36-82d9-3677c8ca993c service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:44.239265 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f0bfa-438b-4f36-82d9-3677c8ca993c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:44.264428 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f0bfa-438b-4f36-82d9-3677c8ca993c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:44.266480 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f0bfa-438b-4f36-82d9-3677c8ca993c service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:44.293432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 856/1696] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:32:44 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:32:44.534363 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2322d355-ab35-4cd0-90e5-5dec5fa43637 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:44.551871 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2322d355-ab35-4cd0-90e5-5dec5fa43637 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:44.562741 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2036d7-9983-4bbf-ac32-0e993bd4c1a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:44.563429 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2036d7-9983-4bbf-ac32-0e993bd4c1a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:44.565795 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2036d7-9983-4bbf-ac32-0e993bd4c1a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:44.566125 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2036d7-9983-4bbf-ac32-0e993bd4c1a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:44.730828 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2036d7-9983-4bbf-ac32-0e993bd4c1a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:44.732031 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2036d7-9983-4bbf-ac32-0e993bd4c1a6 service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:44.771280 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 841/1697] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:32:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:32:45.235800 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f0bfa-438b-4f36-82d9-3677c8ca993c service cinder] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:45.235800 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-437f0bfa-438b-4f36-82d9-3677c8ca993c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:45.457331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:45.527325 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f0bfa-438b-4f36-82d9-3677c8ca993c service cinder] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:45.549251 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d34891-81a9-4468-a449-393717211942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:45.550791 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d34891-81a9-4468-a449-393717211942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:45.552405 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d34891-81a9-4468-a449-393717211942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:45.552843 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d34891-81a9-4468-a449-393717211942 admin admin] PATH_INFO: `/v3/users/33953f2fd2f2469e8668120ea4f9bc92` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:45.558893 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d34891-81a9-4468-a449-393717211942 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33953f2fd2f2469e8668120ea4f9bc92)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:45.562942 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d34891-81a9-4468-a449-393717211942 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:45.705586 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d5d34891-81a9-4468-a449-393717211942 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33953f2fd2f2469e8668120ea4f9bc92'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:45.706683 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d5d34891-81a9-4468-a449-393717211942 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33953f2fd2f2469e8668120ea4f9bc92'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:45.754627 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d5d34891-81a9-4468-a449-393717211942 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33953f2fd2f2469e8668120ea4f9bc92'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:45.824707 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d5d34891-81a9-4468-a449-393717211942 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33953f2fd2f2469e8668120ea4f9bc92'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:45.826143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d5d34891-81a9-4468-a449-393717211942 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33953f2fd2f2469e8668120ea4f9bc92'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:45.877415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 857/1698] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:32:45 2022] DELETE /identity/v3/users/33953f2fd2f2469e8668120ea4f9bc92 => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:45.881666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c6884-e868-4d9c-b378-ceceeb796cf4 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:45.882413 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c6884-e868-4d9c-b378-ceceeb796cf4 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:45.882567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c6884-e868-4d9c-b378-ceceeb796cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:45.986893 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-ba3c6884-e868-4d9c-b378-ceceeb796cf4 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:46.002138 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-ba3c6884-e868-4d9c-b378-ceceeb796cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:46.271939 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 842/1699] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:46.388064 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d34891-81a9-4468-a449-393717211942 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:46.389760 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-d5d34891-81a9-4468-a449-393717211942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:46.600178 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d34891-81a9-4468-a449-393717211942 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:46.613751 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76fe425-91e3-4897-b051-f0876feb9402 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:46.615434 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76fe425-91e3-4897-b051-f0876feb9402 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:46.621508 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76fe425-91e3-4897-b051-f0876feb9402 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:46.622633 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76fe425-91e3-4897-b051-f0876feb9402 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:46.656352 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76fe425-91e3-4897-b051-f0876feb9402 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:46.661542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76fe425-91e3-4897-b051-f0876feb9402 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:46.694901 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 858/1700] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:32:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:32:48.351193 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa38070-e4c0-4164-b26e-ce1e4f3a23e0 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:48.351766 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa38070-e4c0-4164-b26e-ce1e4f3a23e0 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:48.351766 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa38070-e4c0-4164-b26e-ce1e4f3a23e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:48.407171 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-aaa38070-e4c0-4164-b26e-ce1e4f3a23e0 None None] MFA Rules not processed for user `cdcfab461e534465a906c0eef201e423`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:48.408516 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-aaa38070-e4c0-4164-b26e-ce1e4f3a23e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:48.443672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:48.584367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 843/1701] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:48.601745 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76fe425-91e3-4897-b051-f0876feb9402 service neutron] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:48.614831 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-f76fe425-91e3-4897-b051-f0876feb9402 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:48.638132 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c494b3-9b14-466f-b725-cc228b407460 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:48.639525 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c494b3-9b14-466f-b725-cc228b407460 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:48.639928 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c494b3-9b14-466f-b725-cc228b407460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:48.696595 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-98c494b3-9b14-466f-b725-cc228b407460 None None] MFA Rules not processed for user `5085a918c6b74488b3dd70c1fb257381`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:48.698504 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-98c494b3-9b14-466f-b725-cc228b407460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:48.715965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:48.721758 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76fe425-91e3-4897-b051-f0876feb9402 service neutron] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:48.741159 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42116a2b-299a-4237-a6a6-443692ca6bd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:48.741839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42116a2b-299a-4237-a6a6-443692ca6bd4 service nova] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:48.742157 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42116a2b-299a-4237-a6a6-443692ca6bd4 service nova] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:48.742422 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42116a2b-299a-4237-a6a6-443692ca6bd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:48.756175 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42116a2b-299a-4237-a6a6-443692ca6bd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:48.758430 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42116a2b-299a-4237-a6a6-443692ca6bd4 service nova] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:48.776802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 859/1702] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:32:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:32:48.777912 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42116a2b-299a-4237-a6a6-443692ca6bd4 service nova] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:48.779380 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-42116a2b-299a-4237-a6a6-443692ca6bd4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:48.819704 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42116a2b-299a-4237-a6a6-443692ca6bd4 service nova] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:48.836377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3701d-2bdd-4282-b875-0514413faa21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:48.837218 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3701d-2bdd-4282-b875-0514413faa21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:48.837654 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3701d-2bdd-4282-b875-0514413faa21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:48.838099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3701d-2bdd-4282-b875-0514413faa21 admin admin] PATH_INFO: `/v3/projects/278d2e73018e464ab63524b9e4ebd32b` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:48.842063 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3701d-2bdd-4282-b875-0514413faa21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=278d2e73018e464ab63524b9e4ebd32b)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:48.844199 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3701d-2bdd-4282-b875-0514413faa21 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:48.866627 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 844/1703] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:48 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:32:48.877276 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d009f61-4310-4f54-abd5-d8a7fdef406d None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:48.877822 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d009f61-4310-4f54-abd5-d8a7fdef406d None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:48.878282 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d009f61-4310-4f54-abd5-d8a7fdef406d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:48.932294 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-6d009f61-4310-4f54-abd5-d8a7fdef406d None None] MFA Rules not processed for user `5085a918c6b74488b3dd70c1fb257381`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:48.935973 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-6d009f61-4310-4f54-abd5-d8a7fdef406d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:48.971128 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-1cf3701d-2bdd-4282-b875-0514413faa21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '278d2e73018e464ab63524b9e4ebd32b', 'request_id': 'req-1cf3701d-2bdd-4282-b875-0514413faa21'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:48.988903 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-1cf3701d-2bdd-4282-b875-0514413faa21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '278d2e73018e464ab63524b9e4ebd32b', 'request_id': 'req-1cf3701d-2bdd-4282-b875-0514413faa21'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:49.008876 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-1cf3701d-2bdd-4282-b875-0514413faa21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '278d2e73018e464ab63524b9e4ebd32b', 'request_id': 'req-1cf3701d-2bdd-4282-b875-0514413faa21'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:49.036239 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-1cf3701d-2bdd-4282-b875-0514413faa21 admin admin] The token cache is being invalidate because project 278d2e73018e464ab63524b9e4ebd32b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:32:49.036541 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-1cf3701d-2bdd-4282-b875-0514413faa21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:49.037570 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-1cf3701d-2bdd-4282-b875-0514413faa21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:49.041292 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 860/1704] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:32:48 2022] DELETE /identity/v3/projects/278d2e73018e464ab63524b9e4ebd32b => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:49.046079 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c124e80-a4d1-4151-83d9-6331b94b33a7 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:49.046375 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c124e80-a4d1-4151-83d9-6331b94b33a7 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:49.047004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c124e80-a4d1-4151-83d9-6331b94b33a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:49.094014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-8c124e80-a4d1-4151-83d9-6331b94b33a7 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:49.100061 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-8c124e80-a4d1-4151-83d9-6331b94b33a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:49.125718 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 845/1705] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:48 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:49.140503 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d009f61-4310-4f54-abd5-d8a7fdef406d None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:49.142357 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-6d009f61-4310-4f54-abd5-d8a7fdef406d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:49.281915 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:49.341174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d009f61-4310-4f54-abd5-d8a7fdef406d None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:49.379650 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2363c21-e8a0-4b4a-889f-b70baa1b6528 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:49.391022 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 861/1706] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:49.392658 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c76fb-82d4-4ce4-b775-307c6fedbfe1 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:49.395321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c76fb-82d4-4ce4-b775-307c6fedbfe1 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:49.396099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c76fb-82d4-4ce4-b775-307c6fedbfe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:49.396491 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2363c21-e8a0-4b4a-889f-b70baa1b6528 service nova] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:49.398658 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2363c21-e8a0-4b4a-889f-b70baa1b6528 service nova] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:49.398995 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2363c21-e8a0-4b4a-889f-b70baa1b6528 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:49.405318 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-a2363c21-e8a0-4b4a-889f-b70baa1b6528 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:49.433718 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2363c21-e8a0-4b4a-889f-b70baa1b6528 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:49.435571 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2363c21-e8a0-4b4a-889f-b70baa1b6528 service nova] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:49.472379 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 846/1707] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:32:49 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:32:49.487313 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-d74c76fb-82d4-4ce4-b775-307c6fedbfe1 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:49.491850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-d74c76fb-82d4-4ce4-b775-307c6fedbfe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:49.492892 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47c2fa-ec3d-4cbb-8c74-9b8f0d1fcc60 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:49.493182 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47c2fa-ec3d-4cbb-8c74-9b8f0d1fcc60 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:49.493462 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47c2fa-ec3d-4cbb-8c74-9b8f0d1fcc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:49.527168 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 862/1708] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:49.528265 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c76fb-82d4-4ce4-b775-307c6fedbfe1 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:49.559326 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c76fb-82d4-4ce4-b775-307c6fedbfe1 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:49.572206 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a3f3b-e241-4936-b9f7-bfb09ee31d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:49.574212 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a3f3b-e241-4936-b9f7-bfb09ee31d45 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:49.575646 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a3f3b-e241-4936-b9f7-bfb09ee31d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:49.576172 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a3f3b-e241-4936-b9f7-bfb09ee31d45 admin admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:49.578622 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a3f3b-e241-4936-b9f7-bfb09ee31d45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:32:49.580043 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a3f3b-e241-4936-b9f7-bfb09ee31d45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:49.588825 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a3f3b-e241-4936-b9f7-bfb09ee31d45 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:49.674172 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-6f47c2fa-ec3d-4cbb-8c74-9b8f0d1fcc60 None None] MFA Rules not processed for user `9158557f1a2947e2815dc8d586d8b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:49.676116 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-6f47c2fa-ec3d-4cbb-8c74-9b8f0d1fcc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:49.682808 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 863/1709] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:32:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:32:49.684273 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a3f3b-e241-4936-b9f7-bfb09ee31d45 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:49.711467 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a3f3b-e241-4936-b9f7-bfb09ee31d45 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:49.727108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443d551-3fc3-4015-80db-1d1fcebe48d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:49.727736 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443d551-3fc3-4015-80db-1d1fcebe48d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:49.728162 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443d551-3fc3-4015-80db-1d1fcebe48d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:49.728561 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443d551-3fc3-4015-80db-1d1fcebe48d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:49.730210 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e443d551-3fc3-4015-80db-1d1fcebe48d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:49.737450 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e443d551-3fc3-4015-80db-1d1fcebe48d6 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:49.823163 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 864/1710] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:32:49 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:32:49.828724 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443d551-3fc3-4015-80db-1d1fcebe48d6 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:49.959710 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:50.044759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 847/1711] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Sep 26 10:32:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 553 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:50.063675 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443d551-3fc3-4015-80db-1d1fcebe48d6 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:50.098122 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f62331-6a2d-44cd-b3a6-a536ec06e046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:50.098767 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f62331-6a2d-44cd-b3a6-a536ec06e046 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:50.099050 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f62331-6a2d-44cd-b3a6-a536ec06e046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:50.099328 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f62331-6a2d-44cd-b3a6-a536ec06e046 admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:50.100076 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f62331-6a2d-44cd-b3a6-a536ec06e046 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:50.102585 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f62331-6a2d-44cd-b3a6-a536ec06e046 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:50.165738 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 865/1712] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:32:49 2022] POST /identity/v3/users => generated 293 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:32:50.187940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47c2fa-ec3d-4cbb-8c74-9b8f0d1fcc60 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:50.215263 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47c2fa-ec3d-4cbb-8c74-9b8f0d1fcc60 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:50.250840 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1ffee-9064-42b3-bae7-011b23afd92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:50.256735 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1ffee-9064-42b3-bae7-011b23afd92f admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:50.258410 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1ffee-9064-42b3-bae7-011b23afd92f admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:50.260263 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1ffee-9064-42b3-bae7-011b23afd92f admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:50.264892 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f1ffee-9064-42b3-bae7-011b23afd92f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:50.267625 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f1ffee-9064-42b3-bae7-011b23afd92f admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:50.302339 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 848/1713] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:32:50.306288 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f62331-6a2d-44cd-b3a6-a536ec06e046 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:50.342309 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f62331-6a2d-44cd-b3a6-a536ec06e046 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:50.352875 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ea2476-53db-4ce2-b21c-61efe5586312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:50.353454 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea2476-53db-4ce2-b21c-61efe5586312 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:50.353735 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea2476-53db-4ce2-b21c-61efe5586312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:50.354624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea2476-53db-4ce2-b21c-61efe5586312 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:50.355287 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ea2476-53db-4ce2-b21c-61efe5586312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:50.357263 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ea2476-53db-4ce2-b21c-61efe5586312 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:50.382573 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 866/1714] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:32:50.386969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1ffee-9064-42b3-bae7-011b23afd92f admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:50.397698 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ea2476-53db-4ce2-b21c-61efe5586312 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:50.403792 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1ffee-9064-42b3-bae7-011b23afd92f admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:50.405462 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-76ea2476-53db-4ce2-b21c-61efe5586312 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:50.430531 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a78f1-16dc-4e68-8e1f-bab166c8850c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:50.431216 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a78f1-16dc-4e68-8e1f-bab166c8850c admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:50.431496 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a78f1-16dc-4e68-8e1f-bab166c8850c admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:50.431770 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a78f1-16dc-4e68-8e1f-bab166c8850c admin admin] PATH_INFO: `/v3/projects/3728d6a0671648a5aee7b616e649ec3d/users/6f7f10bf82af4ee08d4967fbf62ce78c/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:50.497591 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a78f1-16dc-4e68-8e1f-bab166c8850c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3728d6a0671648a5aee7b616e649ec3d, user_id=6f7f10bf82af4ee08d4967fbf62ce78c, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:50.500332 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a78f1-16dc-4e68-8e1f-bab166c8850c admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:50.554535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 849/1715] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:32:50 2022] PUT /identity/v3/projects/3728d6a0671648a5aee7b616e649ec3d/users/6f7f10bf82af4ee08d4967fbf62ce78c/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:50.567338 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413d70d-894b-4e88-8212-56b63f08a515 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:50.567857 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413d70d-894b-4e88-8212-56b63f08a515 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:50.568312 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413d70d-894b-4e88-8212-56b63f08a515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:50.653716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-7413d70d-894b-4e88-8212-56b63f08a515 None None] MFA Rules not processed for user `6f7f10bf82af4ee08d4967fbf62ce78c`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:50.657594 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-7413d70d-894b-4e88-8212-56b63f08a515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:50.712023 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ea2476-53db-4ce2-b21c-61efe5586312 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:50.858274 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 850/1716] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:32:50.859493 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5cb96a-1cb6-401d-b689-7f9cab123e4c None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:50.862354 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5cb96a-1cb6-401d-b689-7f9cab123e4c None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:50.863756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5cb96a-1cb6-401d-b689-7f9cab123e4c None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:50.866837 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 851/1717] 10.222.0.49 () {54 vars in 912 bytes} [Mon Sep 26 10:32:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:32:50.868564 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5cb96a-1cb6-401d-b689-7f9cab123e4c None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:50.875364 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2f236-0012-4131-9b06-7caf96df9440 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e6be34fc93a4551a83e5b8380188a6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:50.879360 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2f236-0012-4131-9b06-7caf96df9440 service swift] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:50.880528 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2f236-0012-4131-9b06-7caf96df9440 service swift] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:50.881074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2f236-0012-4131-9b06-7caf96df9440 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:50.907933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:50.999985 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:51.069389 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5cb96a-1cb6-401d-b689-7f9cab123e4c None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:51.075420 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2f236-0012-4131-9b06-7caf96df9440 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:51.080316 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2f236-0012-4131-9b06-7caf96df9440 service swift] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:51.084967 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56ca30b-c160-4ae1-ba67-d1b6687db70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:51.085598 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ca30b-c160-4ae1-ba67-d1b6687db70c admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:51.085879 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ca30b-c160-4ae1-ba67-d1b6687db70c admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:51.086189 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ca30b-c160-4ae1-ba67-d1b6687db70c admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:51.086882 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56ca30b-c160-4ae1-ba67-d1b6687db70c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:51.092882 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56ca30b-c160-4ae1-ba67-d1b6687db70c admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:51.132401 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 867/1718] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 10:32:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 733 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 26 10:32:51.135043 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54dd878-be6b-43e7-9185-929e3cfc0e44 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:51.136116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54dd878-be6b-43e7-9185-929e3cfc0e44 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:51.137841 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54dd878-be6b-43e7-9185-929e3cfc0e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:51.158287 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 852/1719] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:32:50 2022] POST /identity/v3/projects => generated 385 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:32:51.161050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56ca30b-c160-4ae1-ba67-d1b6687db70c admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:51.224448 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-e54dd878-be6b-43e7-9185-929e3cfc0e44 None None] MFA Rules not processed for user `f841bf31c43b4333a3d834d38b5197e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:51.227670 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-e54dd878-be6b-43e7-9185-929e3cfc0e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:51.427467 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 868/1720] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Sep 26 10:32:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:51.479212 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56ca30b-c160-4ae1-ba67-d1b6687db70c admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:51.493049 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54dd878-be6b-43e7-9185-929e3cfc0e44 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:51.495319 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-e54dd878-be6b-43e7-9185-929e3cfc0e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:51.495627 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08a54d-3265-42eb-a4b2-dd0fdf6f65a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:51.499607 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08a54d-3265-42eb-a4b2-dd0fdf6f65a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:51.501648 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08a54d-3265-42eb-a4b2-dd0fdf6f65a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:51.502482 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08a54d-3265-42eb-a4b2-dd0fdf6f65a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:51.504285 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08a54d-3265-42eb-a4b2-dd0fdf6f65a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:51.509337 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08a54d-3265-42eb-a4b2-dd0fdf6f65a2 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:51.525023 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54dd878-be6b-43e7-9185-929e3cfc0e44 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:51.536831 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791891d4-6a01-4fdb-b0ac-7958d3767a0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:51.536831 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791891d4-6a01-4fdb-b0ac-7958d3767a0c service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:51.536831 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791891d4-6a01-4fdb-b0ac-7958d3767a0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:51.538306 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791891d4-6a01-4fdb-b0ac-7958d3767a0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:51.571840 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 853/1721] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:32:51 2022] POST /identity/v3/users => generated 299 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:32:51.576945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2bfd2-d39f-4f05-83e4-5bea517e41f8 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:51.580153 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2bfd2-d39f-4f05-83e4-5bea517e41f8 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:51.580325 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791891d4-6a01-4fdb-b0ac-7958d3767a0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:51.581452 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2bfd2-d39f-4f05-83e4-5bea517e41f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:51.582659 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791891d4-6a01-4fdb-b0ac-7958d3767a0c service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:51.608007 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 869/1722] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:32:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:32:51.608913 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791891d4-6a01-4fdb-b0ac-7958d3767a0c service neutron] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:51.660492 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-69d2bfd2-d39f-4f05-83e4-5bea517e41f8 None None] MFA Rules not processed for user `b486a094550541729c19c8713a648243`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:51.661829 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-69d2bfd2-d39f-4f05-83e4-5bea517e41f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:51.694629 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791891d4-6a01-4fdb-b0ac-7958d3767a0c service neutron] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:51.712129 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09ba71-754b-4493-8d14-c4bb9008f8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:51.712830 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09ba71-754b-4493-8d14-c4bb9008f8a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:51.713029 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09ba71-754b-4493-8d14-c4bb9008f8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:51.713504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09ba71-754b-4493-8d14-c4bb9008f8a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:51.714324 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f09ba71-754b-4493-8d14-c4bb9008f8a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:51.718178 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f09ba71-754b-4493-8d14-c4bb9008f8a3 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:51.741377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 870/1723] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:32:51.742542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09ba71-754b-4493-8d14-c4bb9008f8a3 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:51.748447 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-8f09ba71-754b-4493-8d14-c4bb9008f8a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:51.782460 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09ba71-754b-4493-8d14-c4bb9008f8a3 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:51.801606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f66ec9-c210-41f8-a690-cb7816a75e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:51.802289 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f66ec9-c210-41f8-a690-cb7816a75e2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:51.802571 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f66ec9-c210-41f8-a690-cb7816a75e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:51.802847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f66ec9-c210-41f8-a690-cb7816a75e2c admin admin] PATH_INFO: `/v3/users/8545e58f849249dca7f95e68295a3674` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:51.806552 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f66ec9-c210-41f8-a690-cb7816a75e2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8545e58f849249dca7f95e68295a3674)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:51.816367 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f66ec9-c210-41f8-a690-cb7816a75e2c admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:51.855402 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 854/1724] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:51.856452 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2bfd2-d39f-4f05-83e4-5bea517e41f8 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:51.877116 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2bfd2-d39f-4f05-83e4-5bea517e41f8 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:51.890636 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f32214-33c4-47ed-84e5-22ada74f4b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:51.891322 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f32214-33c4-47ed-84e5-22ada74f4b53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:51.893171 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f32214-33c4-47ed-84e5-22ada74f4b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:51.893613 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f32214-33c4-47ed-84e5-22ada74f4b53 admin admin] PATH_INFO: `/v3/projects/3ef9c9eaedfc442d9e8a6e2c4a980633/users/87da9f3646fd4ce4b8cf3e91ec8f14da/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:51.915425 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:51.922660 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-57f66ec9-c210-41f8-a690-cb7816a75e2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8545e58f849249dca7f95e68295a3674'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:51.924336 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-57f66ec9-c210-41f8-a690-cb7816a75e2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8545e58f849249dca7f95e68295a3674'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:51.935462 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f32214-33c4-47ed-84e5-22ada74f4b53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ef9c9eaedfc442d9e8a6e2c4a980633, user_id=87da9f3646fd4ce4b8cf3e91ec8f14da, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:51.940386 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f32214-33c4-47ed-84e5-22ada74f4b53 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:51.946357 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-57f66ec9-c210-41f8-a690-cb7816a75e2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8545e58f849249dca7f95e68295a3674'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:51.978583 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-57f66ec9-c210-41f8-a690-cb7816a75e2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8545e58f849249dca7f95e68295a3674'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:51.991442 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-57f66ec9-c210-41f8-a690-cb7816a75e2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8545e58f849249dca7f95e68295a3674'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:52.015774 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 855/1725] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:32:51 2022] PUT /identity/v3/projects/3ef9c9eaedfc442d9e8a6e2c4a980633/users/87da9f3646fd4ce4b8cf3e91ec8f14da/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:52.021712 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f32214-33c4-47ed-84e5-22ada74f4b53 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:52.029073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-63f32214-33c4-47ed-84e5-22ada74f4b53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:52.054006 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 871/1726] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:32:51 2022] DELETE /identity/v3/users/8545e58f849249dca7f95e68295a3674 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:52.055062 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f66ec9-c210-41f8-a690-cb7816a75e2c admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:52.063299 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-57f66ec9-c210-41f8-a690-cb7816a75e2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:52.180405 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f32214-33c4-47ed-84e5-22ada74f4b53 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:52.229601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:52.362274 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f66ec9-c210-41f8-a690-cb7816a75e2c admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:52.380197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d354854-2254-49fd-b781-a9f38ce67bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:52.381068 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d354854-2254-49fd-b781-a9f38ce67bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:52.381853 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d354854-2254-49fd-b781-a9f38ce67bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:52.381853 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d354854-2254-49fd-b781-a9f38ce67bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:52.382834 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d354854-2254-49fd-b781-a9f38ce67bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:52.387321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d354854-2254-49fd-b781-a9f38ce67bd9 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:52.391743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c025822c-420b-4ff7-8ff5-3609bd6e3d61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:52.395197 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025822c-420b-4ff7-8ff5-3609bd6e3d61 service nova] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:52.395702 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025822c-420b-4ff7-8ff5-3609bd6e3d61 service nova] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:52.399354 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025822c-420b-4ff7-8ff5-3609bd6e3d61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:52.413038 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-c025822c-420b-4ff7-8ff5-3609bd6e3d61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:52.414518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 872/1727] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:32:52 2022] GET /identity/v3/roles => generated 1420 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:32:52.426140 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37678b01-4c05-4d9c-8adc-f4bc622bfbc1 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:52.426631 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37678b01-4c05-4d9c-8adc-f4bc622bfbc1 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:52.427155 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37678b01-4c05-4d9c-8adc-f4bc622bfbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:52.455786 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-37678b01-4c05-4d9c-8adc-f4bc622bfbc1 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:52.459229 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-37678b01-4c05-4d9c-8adc-f4bc622bfbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:52.498582 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 873/1728] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:52.499625 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37678b01-4c05-4d9c-8adc-f4bc622bfbc1 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:52.513243 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37678b01-4c05-4d9c-8adc-f4bc622bfbc1 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:52.521443 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08713c70-c51d-4714-ae13-3677c21122a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:52.522801 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08713c70-c51d-4714-ae13-3677c21122a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:52.523305 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08713c70-c51d-4714-ae13-3677c21122a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:52.523808 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08713c70-c51d-4714-ae13-3677c21122a0 admin admin] PATH_INFO: `/v3/projects/3ef9c9eaedfc442d9e8a6e2c4a980633/users/87da9f3646fd4ce4b8cf3e91ec8f14da/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:52.531873 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08713c70-c51d-4714-ae13-3677c21122a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ef9c9eaedfc442d9e8a6e2c4a980633, user_id=87da9f3646fd4ce4b8cf3e91ec8f14da, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:52.541228 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08713c70-c51d-4714-ae13-3677c21122a0 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:52.572328 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 874/1729] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:32:52 2022] PUT /identity/v3/projects/3ef9c9eaedfc442d9e8a6e2c4a980633/users/87da9f3646fd4ce4b8cf3e91ec8f14da/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:52.573621 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08713c70-c51d-4714-ae13-3677c21122a0 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:52.576408 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-08713c70-c51d-4714-ae13-3677c21122a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:52.588011 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c025822c-420b-4ff7-8ff5-3609bd6e3d61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:52.589695 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c025822c-420b-4ff7-8ff5-3609bd6e3d61 service nova] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:52.628117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 856/1730] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:32:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:32:52.629337 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0530d03d-546f-4f51-9cc5-2e50535c971c None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:52.629696 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0530d03d-546f-4f51-9cc5-2e50535c971c None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:52.638460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0530d03d-546f-4f51-9cc5-2e50535c971c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:52.737862 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-0530d03d-546f-4f51-9cc5-2e50535c971c None None] MFA Rules not processed for user `87da9f3646fd4ce4b8cf3e91ec8f14da`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:52.739285 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-0530d03d-546f-4f51-9cc5-2e50535c971c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:52.753381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08713c70-c51d-4714-ae13-3677c21122a0 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:52.766985 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d320d9bd-11f5-4604-a337-49a79cdb0578 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:52.768324 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320d9bd-11f5-4604-a337-49a79cdb0578 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:52.778184 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320d9bd-11f5-4604-a337-49a79cdb0578 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:52.778184 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320d9bd-11f5-4604-a337-49a79cdb0578 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:52.779680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:52.991153 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 857/1731] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:52 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:53.011005 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0530d03d-546f-4f51-9cc5-2e50535c971c None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:53.012776 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-0530d03d-546f-4f51-9cc5-2e50535c971c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:53.043703 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d320d9bd-11f5-4604-a337-49a79cdb0578 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:53.048803 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d320d9bd-11f5-4604-a337-49a79cdb0578 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:53.080047 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 875/1732] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:32:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:32:53.241293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0530d03d-546f-4f51-9cc5-2e50535c971c None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:53.259593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b298d6bc-1c14-4c42-aae8-1b18f3b03135 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:53.260776 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298d6bc-1c14-4c42-aae8-1b18f3b03135 service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:53.261060 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298d6bc-1c14-4c42-aae8-1b18f3b03135 service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:53.264170 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298d6bc-1c14-4c42-aae8-1b18f3b03135 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:53.290130 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b298d6bc-1c14-4c42-aae8-1b18f3b03135 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:53.292536 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b298d6bc-1c14-4c42-aae8-1b18f3b03135 service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:53.324285 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 858/1733] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:32:53 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:32:53.509599 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9987c81-b7b2-406a-a22f-0be2acd5662d None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:53.510881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9987c81-b7b2-406a-a22f-0be2acd5662d None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:53.511513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9987c81-b7b2-406a-a22f-0be2acd5662d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:53.551288 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b298d6bc-1c14-4c42-aae8-1b18f3b03135 service cinder] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:53.555391 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-e9987c81-b7b2-406a-a22f-0be2acd5662d None None] MFA Rules not processed for user `5085a918c6b74488b3dd70c1fb257381`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:53.556826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-e9987c81-b7b2-406a-a22f-0be2acd5662d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:53.569639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b298d6bc-1c14-4c42-aae8-1b18f3b03135 service cinder] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:53.585980 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7377cfb-6c5f-4c67-a78d-9bd5935ce18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:53.586666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7377cfb-6c5f-4c67-a78d-9bd5935ce18d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:53.586948 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7377cfb-6c5f-4c67-a78d-9bd5935ce18d admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:53.587230 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7377cfb-6c5f-4c67-a78d-9bd5935ce18d admin admin] PATH_INFO: `/v3/users/6f7f10bf82af4ee08d4967fbf62ce78c` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:53.589601 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7377cfb-6c5f-4c67-a78d-9bd5935ce18d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f7f10bf82af4ee08d4967fbf62ce78c)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:53.594783 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7377cfb-6c5f-4c67-a78d-9bd5935ce18d admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:53.695822 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 876/1734] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:53.779584 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-c7377cfb-6c5f-4c67-a78d-9bd5935ce18d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f7f10bf82af4ee08d4967fbf62ce78c'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:53.780270 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-c7377cfb-6c5f-4c67-a78d-9bd5935ce18d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f7f10bf82af4ee08d4967fbf62ce78c'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:53.781070 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-c7377cfb-6c5f-4c67-a78d-9bd5935ce18d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f7f10bf82af4ee08d4967fbf62ce78c'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:53.809006 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-c7377cfb-6c5f-4c67-a78d-9bd5935ce18d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f7f10bf82af4ee08d4967fbf62ce78c'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:53.845090 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-c7377cfb-6c5f-4c67-a78d-9bd5935ce18d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f7f10bf82af4ee08d4967fbf62ce78c'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:53.889336 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 859/1735] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:32:53 2022] DELETE /identity/v3/users/6f7f10bf82af4ee08d4967fbf62ce78c => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:53.891208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6018b0-bf7c-494d-81e4-61bbb68d5127 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:53.891710 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6018b0-bf7c-494d-81e4-61bbb68d5127 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:53.891835 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6018b0-bf7c-494d-81e4-61bbb68d5127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:53.904039 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:53.939770 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-ed6018b0-bf7c-494d-81e4-61bbb68d5127 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:53.940942 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-ed6018b0-bf7c-494d-81e4-61bbb68d5127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:54.153993 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 877/1736] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:54.208905 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7377cfb-6c5f-4c67-a78d-9bd5935ce18d admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:54.210661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-c7377cfb-6c5f-4c67-a78d-9bd5935ce18d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:54.322140 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:54.446676 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7377cfb-6c5f-4c67-a78d-9bd5935ce18d admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:54.463780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e719318-4c30-431f-bf67-2eb94600623d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:54.464535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e719318-4c30-431f-bf67-2eb94600623d service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:54.464956 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e719318-4c30-431f-bf67-2eb94600623d service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:54.465402 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e719318-4c30-431f-bf67-2eb94600623d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:54.487613 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c935621-e562-4e3b-912b-9f217c09e959 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:54.489849 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c935621-e562-4e3b-912b-9f217c09e959 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:54.490559 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c935621-e562-4e3b-912b-9f217c09e959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:54.528718 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e719318-4c30-431f-bf67-2eb94600623d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:54.533469 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e719318-4c30-431f-bf67-2eb94600623d service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:54.598866 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 860/1737] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:32:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:32:54.623602 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-8c935621-e562-4e3b-912b-9f217c09e959 None None] MFA Rules not processed for user `f841bf31c43b4333a3d834d38b5197e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:54.625217 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-8c935621-e562-4e3b-912b-9f217c09e959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:54.661190 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 878/1738] 10.222.0.57 () {58 vars in 1015 bytes} [Mon Sep 26 10:32:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:54.742935 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e719318-4c30-431f-bf67-2eb94600623d service neutron] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:54.763344 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e719318-4c30-431f-bf67-2eb94600623d service neutron] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:54.779622 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded1f5b-19f2-4ca0-8b13-c47de15df543 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:54.780462 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded1f5b-19f2-4ca0-8b13-c47de15df543 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:54.781097 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded1f5b-19f2-4ca0-8b13-c47de15df543 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:54.781720 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded1f5b-19f2-4ca0-8b13-c47de15df543 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:54.806808 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded1f5b-19f2-4ca0-8b13-c47de15df543 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:54.811445 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded1f5b-19f2-4ca0-8b13-c47de15df543 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:54.864225 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 861/1739] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:32:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:32:56.856615 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c935621-e562-4e3b-912b-9f217c09e959 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:56.858094 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-8c935621-e562-4e3b-912b-9f217c09e959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:56.893101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c935621-e562-4e3b-912b-9f217c09e959 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:56.909361 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1deae62-88fc-4a5c-a43a-9cea608a6dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:56.910513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1deae62-88fc-4a5c-a43a-9cea608a6dfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:56.911723 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1deae62-88fc-4a5c-a43a-9cea608a6dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:56.912883 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1deae62-88fc-4a5c-a43a-9cea608a6dfa admin admin] PATH_INFO: `/v3/projects/be07abf31b0b49ebb6fd8ce42e1583f2` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:56.916925 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1deae62-88fc-4a5c-a43a-9cea608a6dfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=be07abf31b0b49ebb6fd8ce42e1583f2)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:56.923829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1deae62-88fc-4a5c-a43a-9cea608a6dfa admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:57.045379 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b82c1e-88ab-4d3c-8c3e-223c69af7bee None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:57.045941 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b82c1e-88ab-4d3c-8c3e-223c69af7bee None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:57.045941 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b82c1e-88ab-4d3c-8c3e-223c69af7bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:57.125174 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c1deae62-88fc-4a5c-a43a-9cea608a6dfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be07abf31b0b49ebb6fd8ce42e1583f2', 'request_id': 'req-c1deae62-88fc-4a5c-a43a-9cea608a6dfa'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:57.128908 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-46b82c1e-88ab-4d3c-8c3e-223c69af7bee None None] MFA Rules not processed for user `cdcfab461e534465a906c0eef201e423`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:32:57.130666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-46b82c1e-88ab-4d3c-8c3e-223c69af7bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:57.149451 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c1deae62-88fc-4a5c-a43a-9cea608a6dfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be07abf31b0b49ebb6fd8ce42e1583f2', 'request_id': 'req-c1deae62-88fc-4a5c-a43a-9cea608a6dfa'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:57.172796 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c1deae62-88fc-4a5c-a43a-9cea608a6dfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be07abf31b0b49ebb6fd8ce42e1583f2', 'request_id': 'req-c1deae62-88fc-4a5c-a43a-9cea608a6dfa'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:57.198629 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c1deae62-88fc-4a5c-a43a-9cea608a6dfa admin admin] The token cache is being invalidate because project be07abf31b0b49ebb6fd8ce42e1583f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:32:57.199756 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c1deae62-88fc-4a5c-a43a-9cea608a6dfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:57.200792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c1deae62-88fc-4a5c-a43a-9cea608a6dfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:57.204007 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 879/1740] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:32:56 2022] DELETE /identity/v3/projects/be07abf31b0b49ebb6fd8ce42e1583f2 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:57.208397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1deae62-88fc-4a5c-a43a-9cea608a6dfa admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:57.212163 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-c1deae62-88fc-4a5c-a43a-9cea608a6dfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:57.310722 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 862/1741] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:32:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:32:57.311739 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b82c1e-88ab-4d3c-8c3e-223c69af7bee None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:57.313184 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-46b82c1e-88ab-4d3c-8c3e-223c69af7bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:57.409715 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1deae62-88fc-4a5c-a43a-9cea608a6dfa admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:57.422688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a5ba1e-6694-42c7-a1b1-ee8ca56bed49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:57.423359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5ba1e-6694-42c7-a1b1-ee8ca56bed49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:57.423667 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5ba1e-6694-42c7-a1b1-ee8ca56bed49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:57.424041 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5ba1e-6694-42c7-a1b1-ee8ca56bed49 admin admin] PATH_INFO: `/v3/projects/3728d6a0671648a5aee7b616e649ec3d` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:57.431013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a5ba1e-6694-42c7-a1b1-ee8ca56bed49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3728d6a0671648a5aee7b616e649ec3d)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:57.433839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a5ba1e-6694-42c7-a1b1-ee8ca56bed49 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:57.580086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:57.608405 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b82c1e-88ab-4d3c-8c3e-223c69af7bee None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:57.624928 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea87af-a5f2-4640-9297-efd58e8881e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:57.625958 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea87af-a5f2-4640-9297-efd58e8881e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:57.626734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea87af-a5f2-4640-9297-efd58e8881e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:57.627158 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea87af-a5f2-4640-9297-efd58e8881e7 admin admin] PATH_INFO: `/v3/users/8c3f8727d5a34c918f07f24b50c0eef2` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:57.628706 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bea87af-a5f2-4640-9297-efd58e8881e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c3f8727d5a34c918f07f24b50c0eef2)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:57.635793 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bea87af-a5f2-4640-9297-efd58e8881e7 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:57.699664 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-18a5ba1e-6694-42c7-a1b1-ee8ca56bed49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3728d6a0671648a5aee7b616e649ec3d', 'request_id': 'req-18a5ba1e-6694-42c7-a1b1-ee8ca56bed49'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:57.718797 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-18a5ba1e-6694-42c7-a1b1-ee8ca56bed49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3728d6a0671648a5aee7b616e649ec3d', 'request_id': 'req-18a5ba1e-6694-42c7-a1b1-ee8ca56bed49'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:57.756864 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-7bea87af-a5f2-4640-9297-efd58e8881e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c3f8727d5a34c918f07f24b50c0eef2'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:57.757245 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-18a5ba1e-6694-42c7-a1b1-ee8ca56bed49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3728d6a0671648a5aee7b616e649ec3d', 'request_id': 'req-18a5ba1e-6694-42c7-a1b1-ee8ca56bed49'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:57.757966 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-7bea87af-a5f2-4640-9297-efd58e8881e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c3f8727d5a34c918f07f24b50c0eef2'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:57.759150 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-7bea87af-a5f2-4640-9297-efd58e8881e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c3f8727d5a34c918f07f24b50c0eef2'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:57.781414 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-7bea87af-a5f2-4640-9297-efd58e8881e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c3f8727d5a34c918f07f24b50c0eef2'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:57.784868 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-18a5ba1e-6694-42c7-a1b1-ee8ca56bed49 admin admin] The token cache is being invalidate because project 3728d6a0671648a5aee7b616e649ec3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:32:57.785160 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-18a5ba1e-6694-42c7-a1b1-ee8ca56bed49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:57.785835 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-18a5ba1e-6694-42c7-a1b1-ee8ca56bed49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:57.786491 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:57.787667 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 880/1742] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:32:57 2022] DELETE /identity/v3/projects/3728d6a0671648a5aee7b616e649ec3d => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:57.798955 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a5ba1e-6694-42c7-a1b1-ee8ca56bed49 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:57.808060 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-18a5ba1e-6694-42c7-a1b1-ee8ca56bed49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:57.820203 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-7bea87af-a5f2-4640-9297-efd58e8881e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c3f8727d5a34c918f07f24b50c0eef2'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:57.852623 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 863/1743] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:32:57 2022] DELETE /identity/v3/users/8c3f8727d5a34c918f07f24b50c0eef2 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:57.902956 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a5ba1e-6694-42c7-a1b1-ee8ca56bed49 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:32:58.067458 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec8898d-8b0a-412f-b3c3-621364a7742d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:32:58.068209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8898d-8b0a-412f-b3c3-621364a7742d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:32:58.073896 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8898d-8b0a-412f-b3c3-621364a7742d admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:32:58.073896 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8898d-8b0a-412f-b3c3-621364a7742d admin admin] PATH_INFO: `/v3/users/87da9f3646fd4ce4b8cf3e91ec8f14da` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:32:58.089393 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec8898d-8b0a-412f-b3c3-621364a7742d admin admin] RBAC: Authorizing `identity:delete_user(user_id=87da9f3646fd4ce4b8cf3e91ec8f14da)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:32:58.097546 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec8898d-8b0a-412f-b3c3-621364a7742d admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:32:58.248735 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-0ec8898d-8b0a-412f-b3c3-621364a7742d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87da9f3646fd4ce4b8cf3e91ec8f14da'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:58.251498 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-0ec8898d-8b0a-412f-b3c3-621364a7742d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87da9f3646fd4ce4b8cf3e91ec8f14da'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:58.271820 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-0ec8898d-8b0a-412f-b3c3-621364a7742d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87da9f3646fd4ce4b8cf3e91ec8f14da'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:58.284416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:32:58.316728 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-0ec8898d-8b0a-412f-b3c3-621364a7742d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87da9f3646fd4ce4b8cf3e91ec8f14da'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:58.317732 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-0ec8898d-8b0a-412f-b3c3-621364a7742d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87da9f3646fd4ce4b8cf3e91ec8f14da'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:32:58.347453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 881/1744] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:32:57 2022] DELETE /identity/v3/users/87da9f3646fd4ce4b8cf3e91ec8f14da => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:32:59.931035 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea87af-a5f2-4640-9297-efd58e8881e7 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:32:59.934128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-7bea87af-a5f2-4640-9297-efd58e8881e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:32:59.957969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea87af-a5f2-4640-9297-efd58e8881e7 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:00.211365 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa695e9-2277-42c4-8218-f192496a946f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:00.214147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa695e9-2277-42c4-8218-f192496a946f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:00.214473 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa695e9-2277-42c4-8218-f192496a946f admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:00.215368 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa695e9-2277-42c4-8218-f192496a946f admin admin] PATH_INFO: `/v3/users/5ac2da43f9c04812ab96e412dcf788c8` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:00.218516 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa695e9-2277-42c4-8218-f192496a946f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ac2da43f9c04812ab96e412dcf788c8)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:00.241779 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa695e9-2277-42c4-8218-f192496a946f admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:00.446358 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6fa695e9-2277-42c4-8218-f192496a946f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ac2da43f9c04812ab96e412dcf788c8'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:00.447661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6fa695e9-2277-42c4-8218-f192496a946f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ac2da43f9c04812ab96e412dcf788c8'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:00.449574 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6fa695e9-2277-42c4-8218-f192496a946f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ac2da43f9c04812ab96e412dcf788c8'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:00.468968 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6fa695e9-2277-42c4-8218-f192496a946f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ac2da43f9c04812ab96e412dcf788c8'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:00.540633 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6fa695e9-2277-42c4-8218-f192496a946f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ac2da43f9c04812ab96e412dcf788c8'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:00.579929 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 864/1745] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:32:59 2022] DELETE /identity/v3/users/5ac2da43f9c04812ab96e412dcf788c8 => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:00.584473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6d356-ec66-4c5f-9858-3ffbe058e727 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:00.585332 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6d356-ec66-4c5f-9858-3ffbe058e727 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:00.585764 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6d356-ec66-4c5f-9858-3ffbe058e727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:00.660108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-12e6d356-ec66-4c5f-9858-3ffbe058e727 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:00.671590 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-12e6d356-ec66-4c5f-9858-3ffbe058e727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:00.959739 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 882/1746] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:00.973922 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8a467-54df-4c0c-a187-e954d864f3f6 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:00.974970 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8a467-54df-4c0c-a187-e954d864f3f6 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:00.975457 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8a467-54df-4c0c-a187-e954d864f3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:01.040231 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-1ce8a467-54df-4c0c-a187-e954d864f3f6 None None] MFA Rules not processed for user `b486a094550541729c19c8713a648243`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:01.042212 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-1ce8a467-54df-4c0c-a187-e954d864f3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:01.060172 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6d356-ec66-4c5f-9858-3ffbe058e727 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:01.061913 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-12e6d356-ec66-4c5f-9858-3ffbe058e727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:01.206248 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:01.227510 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 865/1747] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:33:01.228813 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8a467-54df-4c0c-a187-e954d864f3f6 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:01.232897 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-1ce8a467-54df-4c0c-a187-e954d864f3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:01.262672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8a467-54df-4c0c-a187-e954d864f3f6 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:01.275467 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9781c1c-769d-4614-b9ee-bdcbf68ce326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:01.276679 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9781c1c-769d-4614-b9ee-bdcbf68ce326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:01.276972 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9781c1c-769d-4614-b9ee-bdcbf68ce326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:01.277228 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9781c1c-769d-4614-b9ee-bdcbf68ce326 admin admin] PATH_INFO: `/v3/projects/6f1f454fe7a14b518efade0b0a052afc` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:01.278655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9781c1c-769d-4614-b9ee-bdcbf68ce326 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f1f454fe7a14b518efade0b0a052afc)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:01.282218 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9781c1c-769d-4614-b9ee-bdcbf68ce326 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:01.383970 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6d356-ec66-4c5f-9858-3ffbe058e727 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:01.412508 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2990261-3d2d-49d8-993e-5668eff64004 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:01.421994 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2990261-3d2d-49d8-993e-5668eff64004 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:01.423537 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2990261-3d2d-49d8-993e-5668eff64004 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:01.428051 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2990261-3d2d-49d8-993e-5668eff64004 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:01.478543 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2990261-3d2d-49d8-993e-5668eff64004 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:01.491955 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2990261-3d2d-49d8-993e-5668eff64004 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:01.494852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-d9781c1c-769d-4614-b9ee-bdcbf68ce326 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f1f454fe7a14b518efade0b0a052afc', 'request_id': 'req-d9781c1c-769d-4614-b9ee-bdcbf68ce326'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:01.515596 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-d9781c1c-769d-4614-b9ee-bdcbf68ce326 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f1f454fe7a14b518efade0b0a052afc', 'request_id': 'req-d9781c1c-769d-4614-b9ee-bdcbf68ce326'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:01.524489 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-d9781c1c-769d-4614-b9ee-bdcbf68ce326 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f1f454fe7a14b518efade0b0a052afc', 'request_id': 'req-d9781c1c-769d-4614-b9ee-bdcbf68ce326'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:01.548061 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-d9781c1c-769d-4614-b9ee-bdcbf68ce326 admin admin] The token cache is being invalidate because project 6f1f454fe7a14b518efade0b0a052afc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:33:01.549352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-d9781c1c-769d-4614-b9ee-bdcbf68ce326 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:01.550478 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-d9781c1c-769d-4614-b9ee-bdcbf68ce326 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:01.554087 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 866/1748] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:33:01 2022] DELETE /identity/v3/projects/6f1f454fe7a14b518efade0b0a052afc => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:01.556961 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453b16c-b8eb-406d-8106-351cb42e2dcc None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:01.557247 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453b16c-b8eb-406d-8106-351cb42e2dcc None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:01.557524 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453b16c-b8eb-406d-8106-351cb42e2dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:01.579153 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 883/1749] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:33:01.602210 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-5453b16c-b8eb-406d-8106-351cb42e2dcc None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:01.603499 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-5453b16c-b8eb-406d-8106-351cb42e2dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:01.849721 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 867/1750] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:33:01.859862 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab8deb-da05-4ca3-963d-10f3cae5f379 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:01.860803 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab8deb-da05-4ca3-963d-10f3cae5f379 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:01.861243 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab8deb-da05-4ca3-963d-10f3cae5f379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:01.893342 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-70ab8deb-da05-4ca3-963d-10f3cae5f379 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:01.895694 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-70ab8deb-da05-4ca3-963d-10f3cae5f379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:01.905189 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 884/1751] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:01.909318 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453b16c-b8eb-406d-8106-351cb42e2dcc None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:01.926865 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453b16c-b8eb-406d-8106-351cb42e2dcc None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:01.943539 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a6003-c07a-4d45-8210-77e7277c8aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:01.944331 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a6003-c07a-4d45-8210-77e7277c8aef admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:01.944841 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a6003-c07a-4d45-8210-77e7277c8aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:01.945176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a6003-c07a-4d45-8210-77e7277c8aef admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:01.946249 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a6003-c07a-4d45-8210-77e7277c8aef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:33:01.946773 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a6003-c07a-4d45-8210-77e7277c8aef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:01.949156 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a6003-c07a-4d45-8210-77e7277c8aef admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:01.992275 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 868/1752] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:33:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:33:01.993034 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab8deb-da05-4ca3-963d-10f3cae5f379 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:02.006426 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab8deb-da05-4ca3-963d-10f3cae5f379 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:02.020057 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205b38b-21a8-4846-b540-bdbaa49c7b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:02.022388 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205b38b-21a8-4846-b540-bdbaa49c7b0f admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:02.023690 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205b38b-21a8-4846-b540-bdbaa49c7b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:02.024325 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205b38b-21a8-4846-b540-bdbaa49c7b0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:02.025907 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b205b38b-21a8-4846-b540-bdbaa49c7b0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:02.032734 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b205b38b-21a8-4846-b540-bdbaa49c7b0f admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:02.145070 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 885/1753] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:33:01 2022] POST /identity/v3/projects => generated 375 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:02.147477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a6003-c07a-4d45-8210-77e7277c8aef admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:02.185905 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a6003-c07a-4d45-8210-77e7277c8aef admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:02.366621 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205b38b-21a8-4846-b540-bdbaa49c7b0f admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:02.368926 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-b205b38b-21a8-4846-b540-bdbaa49c7b0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:02.467936 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54722cd-804f-4884-bbd5-94d005065b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:02.470767 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54722cd-804f-4884-bbd5-94d005065b3d admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:02.471198 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54722cd-804f-4884-bbd5-94d005065b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:02.473725 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54722cd-804f-4884-bbd5-94d005065b3d admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:02.477806 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54722cd-804f-4884-bbd5-94d005065b3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:02.490351 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54722cd-804f-4884-bbd5-94d005065b3d admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:02.548026 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:02.561622 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 869/1754] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:33:02 2022] POST /identity/v3/users => generated 288 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:02.566712 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54722cd-804f-4884-bbd5-94d005065b3d admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:02.574960 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:02.595978 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54722cd-804f-4884-bbd5-94d005065b3d admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:02.606519 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d427a3-592f-48c7-80f4-cad4edfb6319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:02.607220 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d427a3-592f-48c7-80f4-cad4edfb6319 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:02.609510 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d427a3-592f-48c7-80f4-cad4edfb6319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:02.609947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d427a3-592f-48c7-80f4-cad4edfb6319 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:02.610975 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d427a3-592f-48c7-80f4-cad4edfb6319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:02.612968 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d427a3-592f-48c7-80f4-cad4edfb6319 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:02.678227 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205b38b-21a8-4846-b540-bdbaa49c7b0f admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:02.686699 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 870/1755] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:02.688706 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d427a3-592f-48c7-80f4-cad4edfb6319 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:02.699941 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b931a-16ff-467c-88ee-3cbfa4a7d51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:02.700916 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b931a-16ff-467c-88ee-3cbfa4a7d51e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:02.701361 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b931a-16ff-467c-88ee-3cbfa4a7d51e admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:02.701914 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b931a-16ff-467c-88ee-3cbfa4a7d51e admin admin] PATH_INFO: `/v3/projects/3ef9c9eaedfc442d9e8a6e2c4a980633` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:02.704744 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1b931a-16ff-467c-88ee-3cbfa4a7d51e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ef9c9eaedfc442d9e8a6e2c4a980633)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:02.707483 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1b931a-16ff-467c-88ee-3cbfa4a7d51e admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:02.735618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d427a3-592f-48c7-80f4-cad4edfb6319 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:02.770761 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043b2a5-859f-4664-a2d6-7cff57d37ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:02.771415 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043b2a5-859f-4664-a2d6-7cff57d37ba3 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:02.771692 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043b2a5-859f-4664-a2d6-7cff57d37ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:02.771968 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043b2a5-859f-4664-a2d6-7cff57d37ba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:02.772618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a043b2a5-859f-4664-a2d6-7cff57d37ba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:02.774264 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a043b2a5-859f-4664-a2d6-7cff57d37ba3 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:02.822297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 871/1756] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:02.824351 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043b2a5-859f-4664-a2d6-7cff57d37ba3 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:02.855393 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043b2a5-859f-4664-a2d6-7cff57d37ba3 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:02.864509 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7f1b931a-16ff-467c-88ee-3cbfa4a7d51e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ef9c9eaedfc442d9e8a6e2c4a980633', 'request_id': 'req-7f1b931a-16ff-467c-88ee-3cbfa4a7d51e'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:02.891565 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7f1b931a-16ff-467c-88ee-3cbfa4a7d51e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ef9c9eaedfc442d9e8a6e2c4a980633', 'request_id': 'req-7f1b931a-16ff-467c-88ee-3cbfa4a7d51e'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:02.916176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7f1b931a-16ff-467c-88ee-3cbfa4a7d51e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ef9c9eaedfc442d9e8a6e2c4a980633', 'request_id': 'req-7f1b931a-16ff-467c-88ee-3cbfa4a7d51e'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:02.952859 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7f1b931a-16ff-467c-88ee-3cbfa4a7d51e admin admin] The token cache is being invalidate because project 3ef9c9eaedfc442d9e8a6e2c4a980633 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:33:02.953179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7f1b931a-16ff-467c-88ee-3cbfa4a7d51e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:02.955021 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7f1b931a-16ff-467c-88ee-3cbfa4a7d51e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:02.958466 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 886/1757] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:33:02 2022] DELETE /identity/v3/projects/3ef9c9eaedfc442d9e8a6e2c4a980633 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:33:02.960892 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbaa254-9ab0-467b-9012-7bf05718623e None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:02.962809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbaa254-9ab0-467b-9012-7bf05718623e None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:02.964429 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbaa254-9ab0-467b-9012-7bf05718623e None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:02.967988 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 887/1758] 10.222.0.49 () {54 vars in 921 bytes} [Mon Sep 26 10:33:02 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:33:02.970791 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0342c-7b41-4497-881c-4878582f2675 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:02.971995 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0342c-7b41-4497-881c-4878582f2675 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:02.972466 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0342c-7b41-4497-881c-4878582f2675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:03.027174 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-a2b0342c-7b41-4497-881c-4878582f2675 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:03.030176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-a2b0342c-7b41-4497-881c-4878582f2675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:03.113288 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b390312-965d-4d5f-bba1-643a3a2fb0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:03.116053 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b390312-965d-4d5f-bba1-643a3a2fb0f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:03.120119 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b390312-965d-4d5f-bba1-643a3a2fb0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:03.120903 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b390312-965d-4d5f-bba1-643a3a2fb0f6 admin admin] PATH_INFO: `/v3/projects/0a61d050e0304da88702b8a62ba1cbd8/users/74226ce13ced49129d822ba92b1a9165/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:03.190311 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b390312-965d-4d5f-bba1-643a3a2fb0f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a61d050e0304da88702b8a62ba1cbd8, user_id=74226ce13ced49129d822ba92b1a9165, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:03.201634 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b390312-965d-4d5f-bba1-643a3a2fb0f6 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:03.260975 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 872/1759] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:02 2022] PUT /identity/v3/projects/0a61d050e0304da88702b8a62ba1cbd8/users/74226ce13ced49129d822ba92b1a9165/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:03.265443 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1d577-0475-4f2d-959c-5be8bb738d47 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:03.265774 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1d577-0475-4f2d-959c-5be8bb738d47 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:03.266088 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1d577-0475-4f2d-959c-5be8bb738d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:03.310196 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 888/1760] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:03.312230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86822f0b-1623-4847-ae77-a372bcb90a06 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:03.313767 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86822f0b-1623-4847-ae77-a372bcb90a06 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:03.313767 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86822f0b-1623-4847-ae77-a372bcb90a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:03.377220 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-46a1d577-0475-4f2d-959c-5be8bb738d47 None None] MFA Rules not processed for user `cee41014d5d043888c6b399710bccab4`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:03.381621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-46a1d577-0475-4f2d-959c-5be8bb738d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:03.426297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-86822f0b-1623-4847-ae77-a372bcb90a06 None None] MFA Rules not processed for user `74226ce13ced49129d822ba92b1a9165`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:03.428012 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-86822f0b-1623-4847-ae77-a372bcb90a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:03.665525 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 889/1761] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:03.667463 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d4a92-7080-46a3-8878-f90abcf2801d None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:03.667748 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d4a92-7080-46a3-8878-f90abcf2801d None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:03.668021 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d4a92-7080-46a3-8878-f90abcf2801d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:03.679461 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 873/1762] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Sep 26 10:33:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:03.680326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1d577-0475-4f2d-959c-5be8bb738d47 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:03.683854 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-46a1d577-0475-4f2d-959c-5be8bb738d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:03.779019 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-ad7d4a92-7080-46a3-8878-f90abcf2801d None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:03.790673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-ad7d4a92-7080-46a3-8878-f90abcf2801d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:03.827657 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:03.940718 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:03.994297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1d577-0475-4f2d-959c-5be8bb738d47 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:04.022052 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd130b-b622-4137-90a5-de22104789b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:04.023418 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd130b-b622-4137-90a5-de22104789b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:04.023418 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd130b-b622-4137-90a5-de22104789b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:04.023705 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd130b-b622-4137-90a5-de22104789b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:04.024297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfd130b-b622-4137-90a5-de22104789b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:04.034186 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 890/1763] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:04.037408 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfd130b-b622-4137-90a5-de22104789b0 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:04.038053 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d4a92-7080-46a3-8878-f90abcf2801d None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:04.058888 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d4a92-7080-46a3-8878-f90abcf2801d None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:04.087600 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd64f19-00c8-44f1-abc5-492c591f8aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:04.088277 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd64f19-00c8-44f1-abc5-492c591f8aed admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:04.088718 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd64f19-00c8-44f1-abc5-492c591f8aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:04.089135 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd64f19-00c8-44f1-abc5-492c591f8aed admin admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:04.089944 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd64f19-00c8-44f1-abc5-492c591f8aed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:33:04.090415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd64f19-00c8-44f1-abc5-492c591f8aed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:04.092187 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd64f19-00c8-44f1-abc5-492c591f8aed admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:04.133624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 891/1764] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:33:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:33:04.138708 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd64f19-00c8-44f1-abc5-492c591f8aed admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:04.160190 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd64f19-00c8-44f1-abc5-492c591f8aed admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:04.171368 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 874/1765] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:33:03 2022] POST /identity/v3/projects => generated 377 bytes in 488 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:04.173901 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd130b-b622-4137-90a5-de22104789b0 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:04.379970 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2277e7e-3ee6-4c35-b23b-feee32a5fad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:04.380602 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2277e7e-3ee6-4c35-b23b-feee32a5fad9 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:04.380883 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2277e7e-3ee6-4c35-b23b-feee32a5fad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:04.381380 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2277e7e-3ee6-4c35-b23b-feee32a5fad9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:04.382736 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2277e7e-3ee6-4c35-b23b-feee32a5fad9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:04.385614 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2277e7e-3ee6-4c35-b23b-feee32a5fad9 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:04.437975 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd130b-b622-4137-90a5-de22104789b0 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:04.458162 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 892/1766] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:33:04 2022] POST /identity/v3/projects => generated 379 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:33:04.458162 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2277e7e-3ee6-4c35-b23b-feee32a5fad9 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:04.459582 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-d2277e7e-3ee6-4c35-b23b-feee32a5fad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:04.486830 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fbd298-6bc2-4536-ad9a-c4dc94ad1be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:04.489784 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fbd298-6bc2-4536-ad9a-c4dc94ad1be7 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:04.491808 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fbd298-6bc2-4536-ad9a-c4dc94ad1be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:04.494943 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fbd298-6bc2-4536-ad9a-c4dc94ad1be7 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:04.498151 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fbd298-6bc2-4536-ad9a-c4dc94ad1be7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:04.514705 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fbd298-6bc2-4536-ad9a-c4dc94ad1be7 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:04.590657 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 875/1767] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:33:04 2022] POST /identity/v3/users => generated 295 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:04.592825 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fbd298-6bc2-4536-ad9a-c4dc94ad1be7 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:04.596378 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-92fbd298-6bc2-4536-ad9a-c4dc94ad1be7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:04.652255 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fbd298-6bc2-4536-ad9a-c4dc94ad1be7 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:04.670698 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2277e7e-3ee6-4c35-b23b-feee32a5fad9 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:04.688580 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595fe7e9-8f93-48b7-8a68-49b88cdbf459 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:04.689363 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595fe7e9-8f93-48b7-8a68-49b88cdbf459 service nova] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:04.689764 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595fe7e9-8f93-48b7-8a68-49b88cdbf459 service nova] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:04.690319 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595fe7e9-8f93-48b7-8a68-49b88cdbf459 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:04.712307 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595fe7e9-8f93-48b7-8a68-49b88cdbf459 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:04.716566 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595fe7e9-8f93-48b7-8a68-49b88cdbf459 service nova] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:04.749094 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 893/1768] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:33:04.749970 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595fe7e9-8f93-48b7-8a68-49b88cdbf459 service nova] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:04.777679 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:04.805311 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c889dc-9520-4577-a863-2193f012dd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:04.805973 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c889dc-9520-4577-a863-2193f012dd0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:04.806287 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c889dc-9520-4577-a863-2193f012dd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:04.806564 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c889dc-9520-4577-a863-2193f012dd0e admin admin] PATH_INFO: `/v3/projects/b3e67f6c643d41349f31ff301d6c9805` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:04.808329 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c889dc-9520-4577-a863-2193f012dd0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3e67f6c643d41349f31ff301d6c9805)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:04.810295 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c889dc-9520-4577-a863-2193f012dd0e admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:04.875686 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595fe7e9-8f93-48b7-8a68-49b88cdbf459 service nova] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:04.881389 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f1953-b8b8-4348-9245-e067891c1840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:04.882042 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f1953-b8b8-4348-9245-e067891c1840 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:04.882333 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f1953-b8b8-4348-9245-e067891c1840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:04.882604 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f1953-b8b8-4348-9245-e067891c1840 admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:04.883280 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818f1953-b8b8-4348-9245-e067891c1840 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:04.886117 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818f1953-b8b8-4348-9245-e067891c1840 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:04.953661 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 894/1769] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:33:04 2022] POST /identity/v3/users => generated 290 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:04.957174 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f1953-b8b8-4348-9245-e067891c1840 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:04.981760 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-46c889dc-9520-4577-a863-2193f012dd0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3e67f6c643d41349f31ff301d6c9805', 'request_id': 'req-46c889dc-9520-4577-a863-2193f012dd0e'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:04.999685 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f1953-b8b8-4348-9245-e067891c1840 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:05.003688 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-46c889dc-9520-4577-a863-2193f012dd0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3e67f6c643d41349f31ff301d6c9805', 'request_id': 'req-46c889dc-9520-4577-a863-2193f012dd0e'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:05.019218 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-46c889dc-9520-4577-a863-2193f012dd0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3e67f6c643d41349f31ff301d6c9805', 'request_id': 'req-46c889dc-9520-4577-a863-2193f012dd0e'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:05.046625 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:05.056074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-46c889dc-9520-4577-a863-2193f012dd0e admin admin] The token cache is being invalidate because project b3e67f6c643d41349f31ff301d6c9805 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:33:05.057874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-46c889dc-9520-4577-a863-2193f012dd0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:05.059682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-46c889dc-9520-4577-a863-2193f012dd0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:05.066495 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 876/1770] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:33:04 2022] DELETE /identity/v3/projects/b3e67f6c643d41349f31ff301d6c9805 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:05.066495 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c889dc-9520-4577-a863-2193f012dd0e admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:05.069581 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-46c889dc-9520-4577-a863-2193f012dd0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:05.145944 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447d738-3274-4892-ac6c-0baed1459dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:05.146711 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447d738-3274-4892-ac6c-0baed1459dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:05.158936 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447d738-3274-4892-ac6c-0baed1459dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:05.159469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447d738-3274-4892-ac6c-0baed1459dd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:05.160535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447d738-3274-4892-ac6c-0baed1459dd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:05.170393 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447d738-3274-4892-ac6c-0baed1459dd1 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:05.205666 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 895/1771] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:05.209310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447d738-3274-4892-ac6c-0baed1459dd1 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:05.213518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-d447d738-3274-4892-ac6c-0baed1459dd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:05.272161 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447d738-3274-4892-ac6c-0baed1459dd1 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:05.308447 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe03fa-e4b0-406b-8630-be7cee588222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:05.309137 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe03fa-e4b0-406b-8630-be7cee588222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:05.313260 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe03fa-e4b0-406b-8630-be7cee588222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:05.313930 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe03fa-e4b0-406b-8630-be7cee588222 admin admin] PATH_INFO: `/v3/users/29f86fb4a4364e6bb4c1e36a119741ff` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:05.316824 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbe03fa-e4b0-406b-8630-be7cee588222 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29f86fb4a4364e6bb4c1e36a119741ff)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:05.319859 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbe03fa-e4b0-406b-8630-be7cee588222 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:05.378585 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c889dc-9520-4577-a863-2193f012dd0e admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:05.413612 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf83ed-8b97-4609-b883-c33c586cf5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:05.419501 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf83ed-8b97-4609-b883-c33c586cf5ec admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:05.420485 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf83ed-8b97-4609-b883-c33c586cf5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:05.421914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf83ed-8b97-4609-b883-c33c586cf5ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:05.422809 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf83ed-8b97-4609-b883-c33c586cf5ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:05.434572 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf83ed-8b97-4609-b883-c33c586cf5ec admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:05.467272 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 877/1772] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:05.469986 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf83ed-8b97-4609-b883-c33c586cf5ec admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:05.474491 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-fcaf83ed-8b97-4609-b883-c33c586cf5ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:05.512449 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf83ed-8b97-4609-b883-c33c586cf5ec admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:05.523940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c28f8e-43e1-45a8-8f64-b9c9402d1cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:05.524900 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c28f8e-43e1-45a8-8f64-b9c9402d1cf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:05.525339 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c28f8e-43e1-45a8-8f64-b9c9402d1cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:05.526537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c28f8e-43e1-45a8-8f64-b9c9402d1cf4 admin admin] PATH_INFO: `/v3/projects/f4e295a6d6104d8483f8eea1f9ba3707/users/1d618382fe2b45a9bb4b77d14c04be83/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:05.537922 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7dbe03fa-e4b0-406b-8630-be7cee588222 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29f86fb4a4364e6bb4c1e36a119741ff'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:05.541176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7dbe03fa-e4b0-406b-8630-be7cee588222 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29f86fb4a4364e6bb4c1e36a119741ff'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:05.553926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c28f8e-43e1-45a8-8f64-b9c9402d1cf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4e295a6d6104d8483f8eea1f9ba3707, user_id=1d618382fe2b45a9bb4b77d14c04be83, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:05.563964 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c28f8e-43e1-45a8-8f64-b9c9402d1cf4 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:05.577644 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7dbe03fa-e4b0-406b-8630-be7cee588222 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29f86fb4a4364e6bb4c1e36a119741ff'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:05.593931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 878/1773] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:05 2022] PUT /identity/v3/projects/f4e295a6d6104d8483f8eea1f9ba3707/users/1d618382fe2b45a9bb4b77d14c04be83/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:05.596084 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c28f8e-43e1-45a8-8f64-b9c9402d1cf4 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:05.601953 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-03c28f8e-43e1-45a8-8f64-b9c9402d1cf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:05.623902 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7dbe03fa-e4b0-406b-8630-be7cee588222 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29f86fb4a4364e6bb4c1e36a119741ff'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:05.625396 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7dbe03fa-e4b0-406b-8630-be7cee588222 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29f86fb4a4364e6bb4c1e36a119741ff'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:05.662407 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 896/1774] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:33:05 2022] DELETE /identity/v3/users/29f86fb4a4364e6bb4c1e36a119741ff => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:05.683022 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe03fa-e4b0-406b-8630-be7cee588222 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:05.687093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-7dbe03fa-e4b0-406b-8630-be7cee588222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:05.707484 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:05.760054 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c28f8e-43e1-45a8-8f64-b9c9402d1cf4 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:05.900818 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe03fa-e4b0-406b-8630-be7cee588222 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:05.913086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a162e064-b986-46b1-93c9-6632d4c4cb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:05.914704 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162e064-b986-46b1-93c9-6632d4c4cb70 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:05.915229 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162e064-b986-46b1-93c9-6632d4c4cb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:05.917648 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162e064-b986-46b1-93c9-6632d4c4cb70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:05.918411 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a162e064-b986-46b1-93c9-6632d4c4cb70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:05.921250 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a162e064-b986-46b1-93c9-6632d4c4cb70 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:05.944742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 897/1775] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:05.946359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a162e064-b986-46b1-93c9-6632d4c4cb70 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:05.954075 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c52d5a-a50c-4886-af2b-663182dd1549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:05.955067 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c52d5a-a50c-4886-af2b-663182dd1549 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:05.956021 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c52d5a-a50c-4886-af2b-663182dd1549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:05.956515 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c52d5a-a50c-4886-af2b-663182dd1549 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:05.958173 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c52d5a-a50c-4886-af2b-663182dd1549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:05.960402 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a162e064-b986-46b1-93c9-6632d4c4cb70 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:05.969675 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c52d5a-a50c-4886-af2b-663182dd1549 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:05.973314 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f40aa-96e2-4c8a-90ea-9df0c04b7c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:05.975290 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f40aa-96e2-4c8a-90ea-9df0c04b7c21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:05.976479 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f40aa-96e2-4c8a-90ea-9df0c04b7c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:05.978636 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f40aa-96e2-4c8a-90ea-9df0c04b7c21 admin admin] PATH_INFO: `/v3/projects/f4e295a6d6104d8483f8eea1f9ba3707/users/1d618382fe2b45a9bb4b77d14c04be83/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:05.984268 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f40aa-96e2-4c8a-90ea-9df0c04b7c21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4e295a6d6104d8483f8eea1f9ba3707, user_id=1d618382fe2b45a9bb4b77d14c04be83, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:05.993065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f40aa-96e2-4c8a-90ea-9df0c04b7c21 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:06.015199 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 879/1776] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:06.040801 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c52d5a-a50c-4886-af2b-663182dd1549 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:06.051494 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 898/1777] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:05 2022] PUT /identity/v3/projects/f4e295a6d6104d8483f8eea1f9ba3707/users/1d618382fe2b45a9bb4b77d14c04be83/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:06.066412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f8514-c6f7-407c-b479-968f247a715f None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:06.066412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f8514-c6f7-407c-b479-968f247a715f None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:06.066616 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f8514-c6f7-407c-b479-968f247a715f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:06.082145 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c52d5a-a50c-4886-af2b-663182dd1549 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:06.125482 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:06.128383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-1c5f8514-c6f7-407c-b479-968f247a715f None None] MFA Rules not processed for user `1d618382fe2b45a9bb4b77d14c04be83`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:06.129729 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-1c5f8514-c6f7-407c-b479-968f247a715f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:06.293640 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 899/1778] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:06 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:33:06.306327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f8514-c6f7-407c-b479-968f247a715f None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:06.308469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-1c5f8514-c6f7-407c-b479-968f247a715f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:06.323691 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797df13-daea-43c1-af5b-4fcb7823ee9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:06.324371 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797df13-daea-43c1-af5b-4fcb7823ee9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:06.324650 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797df13-daea-43c1-af5b-4fcb7823ee9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:06.324925 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797df13-daea-43c1-af5b-4fcb7823ee9f admin admin] PATH_INFO: `/v3/projects/07dc98af8ae14cb49de1695c37d75af3/users/7073c61ab3e24574a95e30c59918ea54/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:06.386848 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c797df13-daea-43c1-af5b-4fcb7823ee9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=07dc98af8ae14cb49de1695c37d75af3, user_id=7073c61ab3e24574a95e30c59918ea54, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:06.421490 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c797df13-daea-43c1-af5b-4fcb7823ee9f admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:06.470169 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 880/1779] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:06 2022] PUT /identity/v3/projects/07dc98af8ae14cb49de1695c37d75af3/users/7073c61ab3e24574a95e30c59918ea54/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:06.484449 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f8514-c6f7-407c-b479-968f247a715f None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:06.500675 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c21984-71e9-4711-8579-abf6bafc1d40 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:06.501900 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c21984-71e9-4711-8579-abf6bafc1d40 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:06.505811 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c21984-71e9-4711-8579-abf6bafc1d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:06.640844 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:06.672011 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-e5c21984-71e9-4711-8579-abf6bafc1d40 None None] MFA Rules not processed for user `7073c61ab3e24574a95e30c59918ea54`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:06.677645 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6ee4e-67d6-41a3-be56-27eef265bfdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:06.678333 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6ee4e-67d6-41a3-be56-27eef265bfdc service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:06.678619 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6ee4e-67d6-41a3-be56-27eef265bfdc service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:06.678894 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6ee4e-67d6-41a3-be56-27eef265bfdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:06.679864 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-e5c21984-71e9-4711-8579-abf6bafc1d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:06.681144 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-06e6ee4e-67d6-41a3-be56-27eef265bfdc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:06.836818 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6ee4e-67d6-41a3-be56-27eef265bfdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:06.842597 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6ee4e-67d6-41a3-be56-27eef265bfdc service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:06.869534 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 900/1780] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:33:07.067446 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 881/1781] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 567 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:07.068742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6ee4e-67d6-41a3-be56-27eef265bfdc service cinder] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:07.225298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6ee4e-67d6-41a3-be56-27eef265bfdc service cinder] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:07.253564 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0aabe3-2fcf-4afe-80f1-f3d9fcf3472c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:07.254172 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0aabe3-2fcf-4afe-80f1-f3d9fcf3472c admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:07.254473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0aabe3-2fcf-4afe-80f1-f3d9fcf3472c admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:07.254734 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0aabe3-2fcf-4afe-80f1-f3d9fcf3472c admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:07.255435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0aabe3-2fcf-4afe-80f1-f3d9fcf3472c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:07.257501 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0aabe3-2fcf-4afe-80f1-f3d9fcf3472c admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:07.331997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 901/1782] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:33:07 2022] POST /identity/v3/projects => generated 381 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:07.336436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c21984-71e9-4711-8579-abf6bafc1d40 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:07.360322 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c21984-71e9-4711-8579-abf6bafc1d40 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:07.585572 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e602ce8b-4210-414c-96bf-cfc8d9eba761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:07.587090 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e602ce8b-4210-414c-96bf-cfc8d9eba761 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:07.587553 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e602ce8b-4210-414c-96bf-cfc8d9eba761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:07.589178 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e602ce8b-4210-414c-96bf-cfc8d9eba761 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:07.590446 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e602ce8b-4210-414c-96bf-cfc8d9eba761 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:07.593445 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e602ce8b-4210-414c-96bf-cfc8d9eba761 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:07.657861 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 882/1783] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:33:07 2022] POST /identity/v3/users => generated 297 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:07.665572 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0aabe3-2fcf-4afe-80f1-f3d9fcf3472c admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:07.681324 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0aabe3-2fcf-4afe-80f1-f3d9fcf3472c admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:07.688407 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ae82e-d15b-4926-a766-e922d46cb902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:07.689852 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ae82e-d15b-4926-a766-e922d46cb902 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:07.690400 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ae82e-d15b-4926-a766-e922d46cb902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:07.690783 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ae82e-d15b-4926-a766-e922d46cb902 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:07.691685 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ae82e-d15b-4926-a766-e922d46cb902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:07.693532 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ae82e-d15b-4926-a766-e922d46cb902 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:07.722001 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 902/1784] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:07.723644 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e602ce8b-4210-414c-96bf-cfc8d9eba761 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:07.747566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e602ce8b-4210-414c-96bf-cfc8d9eba761 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:07.759501 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e8cff-b6ee-411c-9b01-43a8607cc417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:07.760880 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e8cff-b6ee-411c-9b01-43a8607cc417 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:07.761190 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e8cff-b6ee-411c-9b01-43a8607cc417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:07.761456 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e8cff-b6ee-411c-9b01-43a8607cc417 admin admin] PATH_INFO: `/v3/projects/f9728e815fe04468803df2c9579ba756/users/38167209ee354f6cafca82f353948b94/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:07.783779 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:07.793006 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137e8cff-b6ee-411c-9b01-43a8607cc417 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9728e815fe04468803df2c9579ba756, user_id=38167209ee354f6cafca82f353948b94, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:07.801348 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137e8cff-b6ee-411c-9b01-43a8607cc417 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:07.849453 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 883/1785] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:07 2022] PUT /identity/v3/projects/f9728e815fe04468803df2c9579ba756/users/38167209ee354f6cafca82f353948b94/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:07.853928 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ae82e-d15b-4926-a766-e922d46cb902 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:07.902596 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ae82e-d15b-4926-a766-e922d46cb902 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:08.086767 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad51b4-c57a-4c3e-b023-da19dd08a82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:08.087955 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad51b4-c57a-4c3e-b023-da19dd08a82f admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:08.087955 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad51b4-c57a-4c3e-b023-da19dd08a82f admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:08.089266 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad51b4-c57a-4c3e-b023-da19dd08a82f admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:08.090544 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad51b4-c57a-4c3e-b023-da19dd08a82f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:08.093559 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad51b4-c57a-4c3e-b023-da19dd08a82f admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:08.096859 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:08.118946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 903/1786] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:08.123890 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e8cff-b6ee-411c-9b01-43a8607cc417 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:08.138969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e8cff-b6ee-411c-9b01-43a8607cc417 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:08.144947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef165f-09d4-4d5a-9069-fdd2583b6bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:08.145904 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef165f-09d4-4d5a-9069-fdd2583b6bf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:08.146297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef165f-09d4-4d5a-9069-fdd2583b6bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:08.146572 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef165f-09d4-4d5a-9069-fdd2583b6bf5 admin admin] PATH_INFO: `/v3/projects/f9728e815fe04468803df2c9579ba756/users/38167209ee354f6cafca82f353948b94/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:08.149196 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ef165f-09d4-4d5a-9069-fdd2583b6bf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9728e815fe04468803df2c9579ba756, user_id=38167209ee354f6cafca82f353948b94, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:08.152194 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ef165f-09d4-4d5a-9069-fdd2583b6bf5 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:08.192493 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 884/1787] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:08 2022] PUT /identity/v3/projects/f9728e815fe04468803df2c9579ba756/users/38167209ee354f6cafca82f353948b94/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:08.203417 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b5683-990f-4c29-a2d5-606a22e29c19 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:08.203946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b5683-990f-4c29-a2d5-606a22e29c19 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:08.204380 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b5683-990f-4c29-a2d5-606a22e29c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:08.267111 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-d38b5683-990f-4c29-a2d5-606a22e29c19 None None] MFA Rules not processed for user `38167209ee354f6cafca82f353948b94`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:08.272397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-d38b5683-990f-4c29-a2d5-606a22e29c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:08.436154 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 904/1788] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:08 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:33:08.447818 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef165f-09d4-4d5a-9069-fdd2583b6bf5 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:08.450847 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-98ef165f-09d4-4d5a-9069-fdd2583b6bf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:08.591115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef165f-09d4-4d5a-9069-fdd2583b6bf5 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:08.599713 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497580f8-94af-4269-b600-c88302ad337b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:08.600596 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497580f8-94af-4269-b600-c88302ad337b service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:08.601523 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497580f8-94af-4269-b600-c88302ad337b service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:08.602769 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497580f8-94af-4269-b600-c88302ad337b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:08.617394 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497580f8-94af-4269-b600-c88302ad337b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:08.619141 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497580f8-94af-4269-b600-c88302ad337b service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:08.629804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 885/1789] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:08 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:33:08.742265 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b5683-990f-4c29-a2d5-606a22e29c19 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:08.742265 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-d38b5683-990f-4c29-a2d5-606a22e29c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:08.859266 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b5683-990f-4c29-a2d5-606a22e29c19 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:08.865204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c84370-8e66-4d1e-8c05-cc17557c3df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:08.865984 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c84370-8e66-4d1e-8c05-cc17557c3df9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:08.866313 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c84370-8e66-4d1e-8c05-cc17557c3df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:08.866904 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c84370-8e66-4d1e-8c05-cc17557c3df9 admin admin] PATH_INFO: `/v3/projects/e899bfdb17624901b92860a5150e1454` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:08.868642 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c84370-8e66-4d1e-8c05-cc17557c3df9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e899bfdb17624901b92860a5150e1454)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:08.871083 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c84370-8e66-4d1e-8c05-cc17557c3df9 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:08.935559 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-05c84370-8e66-4d1e-8c05-cc17557c3df9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e899bfdb17624901b92860a5150e1454', 'request_id': 'req-05c84370-8e66-4d1e-8c05-cc17557c3df9'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:08.943793 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-05c84370-8e66-4d1e-8c05-cc17557c3df9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e899bfdb17624901b92860a5150e1454', 'request_id': 'req-05c84370-8e66-4d1e-8c05-cc17557c3df9'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:08.953945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-05c84370-8e66-4d1e-8c05-cc17557c3df9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e899bfdb17624901b92860a5150e1454', 'request_id': 'req-05c84370-8e66-4d1e-8c05-cc17557c3df9'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:08.973904 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-05c84370-8e66-4d1e-8c05-cc17557c3df9 admin admin] The token cache is being invalidate because project e899bfdb17624901b92860a5150e1454 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:33:08.974659 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-05c84370-8e66-4d1e-8c05-cc17557c3df9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:08.975951 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-05c84370-8e66-4d1e-8c05-cc17557c3df9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:08.978397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 905/1790] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:33:08 2022] DELETE /identity/v3/projects/e899bfdb17624901b92860a5150e1454 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:08.983176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed2c81-04a6-43a5-ba84-ed2fd4e57c14 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:08.983489 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed2c81-04a6-43a5-ba84-ed2fd4e57c14 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:08.983889 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed2c81-04a6-43a5-ba84-ed2fd4e57c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:09.033778 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-f3ed2c81-04a6-43a5-ba84-ed2fd4e57c14 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:09.035682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-f3ed2c81-04a6-43a5-ba84-ed2fd4e57c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:09.138297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 886/1791] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:09.139391 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecdbee2-ba82-4306-8e38-04fef846ce2d None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:09.139698 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecdbee2-ba82-4306-8e38-04fef846ce2d None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:09.139973 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecdbee2-ba82-4306-8e38-04fef846ce2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:09.177907 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-5ecdbee2-ba82-4306-8e38-04fef846ce2d None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:09.184170 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-5ecdbee2-ba82-4306-8e38-04fef846ce2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:09.200562 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 906/1792] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:09.204810 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed2c81-04a6-43a5-ba84-ed2fd4e57c14 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:09.219932 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed2c81-04a6-43a5-ba84-ed2fd4e57c14 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:09.228471 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecdbee2-ba82-4306-8e38-04fef846ce2d None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:09.229948 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-5ecdbee2-ba82-4306-8e38-04fef846ce2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:09.246253 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851912a-a0dc-4dda-bc58-a89ae2f51c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:09.246253 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851912a-a0dc-4dda-bc58-a89ae2f51c31 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:09.246253 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851912a-a0dc-4dda-bc58-a89ae2f51c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:09.246253 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851912a-a0dc-4dda-bc58-a89ae2f51c31 admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:09.246622 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851912a-a0dc-4dda-bc58-a89ae2f51c31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:33:09.246622 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851912a-a0dc-4dda-bc58-a89ae2f51c31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:09.246622 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851912a-a0dc-4dda-bc58-a89ae2f51c31 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:09.256374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecdbee2-ba82-4306-8e38-04fef846ce2d None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:09.265770 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a44e6-4bb1-4084-9eba-34ee9728194a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:09.265770 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a44e6-4bb1-4084-9eba-34ee9728194a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:09.265770 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a44e6-4bb1-4084-9eba-34ee9728194a admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:09.265770 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a44e6-4bb1-4084-9eba-34ee9728194a admin admin] PATH_INFO: `/v3/users/7073c61ab3e24574a95e30c59918ea54` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:09.266197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a44e6-4bb1-4084-9eba-34ee9728194a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7073c61ab3e24574a95e30c59918ea54)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:09.268082 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a44e6-4bb1-4084-9eba-34ee9728194a admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:09.270430 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:09.281500 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 887/1793] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:33:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:33:09.286463 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851912a-a0dc-4dda-bc58-a89ae2f51c31 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:09.302795 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851912a-a0dc-4dda-bc58-a89ae2f51c31 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:09.309727 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa7a39-0527-4dc4-8877-b77bbf66998f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:09.310759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa7a39-0527-4dc4-8877-b77bbf66998f admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:09.311185 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa7a39-0527-4dc4-8877-b77bbf66998f admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:09.311668 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa7a39-0527-4dc4-8877-b77bbf66998f admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:09.312927 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa7a39-0527-4dc4-8877-b77bbf66998f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:09.316478 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa7a39-0527-4dc4-8877-b77bbf66998f admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:09.360952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 888/1794] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:33:09 2022] POST /identity/v3/projects => generated 363 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:33:09.365865 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa7a39-0527-4dc4-8877-b77bbf66998f admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:09.376211 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-688a44e6-4bb1-4084-9eba-34ee9728194a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7073c61ab3e24574a95e30c59918ea54'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:09.377567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-688a44e6-4bb1-4084-9eba-34ee9728194a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7073c61ab3e24574a95e30c59918ea54'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:09.387620 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-688a44e6-4bb1-4084-9eba-34ee9728194a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7073c61ab3e24574a95e30c59918ea54'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:09.421779 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-688a44e6-4bb1-4084-9eba-34ee9728194a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7073c61ab3e24574a95e30c59918ea54'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:09.422975 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-688a44e6-4bb1-4084-9eba-34ee9728194a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7073c61ab3e24574a95e30c59918ea54'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:09.437079 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:09.441394 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 907/1795] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:33:09 2022] DELETE /identity/v3/users/7073c61ab3e24574a95e30c59918ea54 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:09.446953 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ff43c-52e8-4e01-863f-5b3952677c19 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:09.447055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ff43c-52e8-4e01-863f-5b3952677c19 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:09.447373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ff43c-52e8-4e01-863f-5b3952677c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:09.462496 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa7a39-0527-4dc4-8877-b77bbf66998f admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:09.479941 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-640ff43c-52e8-4e01-863f-5b3952677c19 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:09.481336 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-640ff43c-52e8-4e01-863f-5b3952677c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:09.537519 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4d117-d085-48c1-b9b4-42094f658f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:09.538410 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4d117-d085-48c1-b9b4-42094f658f0a admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:09.538693 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4d117-d085-48c1-b9b4-42094f658f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:09.539117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4d117-d085-48c1-b9b4-42094f658f0a admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:09.539765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f4d117-d085-48c1-b9b4-42094f658f0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:09.542000 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f4d117-d085-48c1-b9b4-42094f658f0a admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:09.571323 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 889/1796] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:33:09 2022] POST /identity/v3/users => generated 282 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:09.577424 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4d117-d085-48c1-b9b4-42094f658f0a admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:09.580863 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-78f4d117-d085-48c1-b9b4-42094f658f0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:09.600399 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4d117-d085-48c1-b9b4-42094f658f0a admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:09.606288 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b410de3-4672-472e-a26e-80c3bc3ea254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:09.606915 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b410de3-4672-472e-a26e-80c3bc3ea254 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:09.607194 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b410de3-4672-472e-a26e-80c3bc3ea254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:09.607472 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b410de3-4672-472e-a26e-80c3bc3ea254 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:09.608185 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b410de3-4672-472e-a26e-80c3bc3ea254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:09.609791 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b410de3-4672-472e-a26e-80c3bc3ea254 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:09.610854 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 908/1797] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:09.617206 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ff43c-52e8-4e01-863f-5b3952677c19 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:09.618725 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-640ff43c-52e8-4e01-863f-5b3952677c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:09.627002 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 890/1798] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:09.630911 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b410de3-4672-472e-a26e-80c3bc3ea254 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:09.642310 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b410de3-4672-472e-a26e-80c3bc3ea254 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:09.647324 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622affe6-ba52-4ec3-b07f-62d01cb5b141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:09.647940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622affe6-ba52-4ec3-b07f-62d01cb5b141 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:09.648228 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622affe6-ba52-4ec3-b07f-62d01cb5b141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:09.648504 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622affe6-ba52-4ec3-b07f-62d01cb5b141 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:09.649145 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622affe6-ba52-4ec3-b07f-62d01cb5b141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:09.650975 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622affe6-ba52-4ec3-b07f-62d01cb5b141 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:09.663142 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 891/1799] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:33:09.667694 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622affe6-ba52-4ec3-b07f-62d01cb5b141 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:09.679531 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622affe6-ba52-4ec3-b07f-62d01cb5b141 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:09.687654 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcc12a1-7d8d-42b9-9501-eb24c86bfe6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:09.688474 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc12a1-7d8d-42b9-9501-eb24c86bfe6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:09.688856 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc12a1-7d8d-42b9-9501-eb24c86bfe6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:09.689224 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc12a1-7d8d-42b9-9501-eb24c86bfe6d admin admin] PATH_INFO: `/v3/projects/1cdadf27e69143d1b4a215c15f9c57b2/users/28e32d81b52c4e71bdae84b3e68fbecd/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:09.691115 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ff43c-52e8-4e01-863f-5b3952677c19 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:09.694802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f41b5a-9387-4c0e-b959-7bee2f45f002 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:09.695413 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f41b5a-9387-4c0e-b959-7bee2f45f002 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:09.695693 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f41b5a-9387-4c0e-b959-7bee2f45f002 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:09.695967 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f41b5a-9387-4c0e-b959-7bee2f45f002 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:09.705785 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f41b5a-9387-4c0e-b959-7bee2f45f002 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:09.707167 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f41b5a-9387-4c0e-b959-7bee2f45f002 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:09.713596 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcc12a1-7d8d-42b9-9501-eb24c86bfe6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cdadf27e69143d1b4a215c15f9c57b2, user_id=28e32d81b52c4e71bdae84b3e68fbecd, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:09.716601 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcc12a1-7d8d-42b9-9501-eb24c86bfe6d admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:09.717529 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 909/1800] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:33:09.737339 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 892/1801] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:09 2022] PUT /identity/v3/projects/1cdadf27e69143d1b4a215c15f9c57b2/users/28e32d81b52c4e71bdae84b3e68fbecd/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:33:09.748911 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340415f-5ed6-4c21-9dea-14e820cd5fc8 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:09.748911 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340415f-5ed6-4c21-9dea-14e820cd5fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:09.748911 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340415f-5ed6-4c21-9dea-14e820cd5fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:09.785762 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-8340415f-5ed6-4c21-9dea-14e820cd5fc8 None None] MFA Rules not processed for user `28e32d81b52c4e71bdae84b3e68fbecd`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:09.787538 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-8340415f-5ed6-4c21-9dea-14e820cd5fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:09.799331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:09.885544 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 910/1802] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:09.886548 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcc12a1-7d8d-42b9-9501-eb24c86bfe6d admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:09.971466 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcc12a1-7d8d-42b9-9501-eb24c86bfe6d admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:09.976843 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b769167-e861-4f01-9dc0-85dbabd645d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:09.977532 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b769167-e861-4f01-9dc0-85dbabd645d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:09.977809 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b769167-e861-4f01-9dc0-85dbabd645d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:09.978184 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b769167-e861-4f01-9dc0-85dbabd645d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:09.979866 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b769167-e861-4f01-9dc0-85dbabd645d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:09.982183 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b769167-e861-4f01-9dc0-85dbabd645d5 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:10.023878 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 893/1803] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:33:09 2022] POST /identity/v3/projects => generated 365 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:33:10.024995 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340415f-5ed6-4c21-9dea-14e820cd5fc8 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:10.038276 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340415f-5ed6-4c21-9dea-14e820cd5fc8 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:10.117914 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbc827-5aca-4409-9d3f-2d17008aa0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:10.118681 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbc827-5aca-4409-9d3f-2d17008aa0bf admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:10.118961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbc827-5aca-4409-9d3f-2d17008aa0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:10.119252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbc827-5aca-4409-9d3f-2d17008aa0bf admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:10.119950 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebbc827-5aca-4409-9d3f-2d17008aa0bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:10.121932 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebbc827-5aca-4409-9d3f-2d17008aa0bf admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:10.130894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b769167-e861-4f01-9dc0-85dbabd645d5 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:10.133853 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-3b769167-e861-4f01-9dc0-85dbabd645d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:10.152741 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 911/1804] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:33:10 2022] POST /identity/v3/users => generated 289 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:10.157043 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbc827-5aca-4409-9d3f-2d17008aa0bf admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:10.165160 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b769167-e861-4f01-9dc0-85dbabd645d5 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:10.168605 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbc827-5aca-4409-9d3f-2d17008aa0bf admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:10.175716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba6309-c5ff-4ed2-b290-52b0950caa36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:10.176358 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba6309-c5ff-4ed2-b290-52b0950caa36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:10.176637 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba6309-c5ff-4ed2-b290-52b0950caa36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:10.176914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba6309-c5ff-4ed2-b290-52b0950caa36 admin admin] PATH_INFO: `/v3/projects/07dc98af8ae14cb49de1695c37d75af3` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:10.178173 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba6309-c5ff-4ed2-b290-52b0950caa36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07dc98af8ae14cb49de1695c37d75af3)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:10.178761 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dccdad-a5f9-41c4-b7ed-7e2a5681a2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:10.179381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dccdad-a5f9-41c4-b7ed-7e2a5681a2e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:10.179661 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dccdad-a5f9-41c4-b7ed-7e2a5681a2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:10.179932 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dccdad-a5f9-41c4-b7ed-7e2a5681a2e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:10.180588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dccdad-a5f9-41c4-b7ed-7e2a5681a2e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:10.181062 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba6309-c5ff-4ed2-b290-52b0950caa36 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:10.182290 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dccdad-a5f9-41c4-b7ed-7e2a5681a2e8 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:10.193470 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 912/1805] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:33:10.197804 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dccdad-a5f9-41c4-b7ed-7e2a5681a2e8 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:10.203909 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:10.209342 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dccdad-a5f9-41c4-b7ed-7e2a5681a2e8 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:10.216670 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69f4b2-86c1-44db-88cd-ce0dc329c882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:10.217405 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69f4b2-86c1-44db-88cd-ce0dc329c882 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:10.217640 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69f4b2-86c1-44db-88cd-ce0dc329c882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:10.217903 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69f4b2-86c1-44db-88cd-ce0dc329c882 admin admin] PATH_INFO: `/v3/projects/7d1ae0a54bc0420182194d607a5b85ee/users/626e3fff43c949a19327fb1df621ee21/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:10.237390 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a69f4b2-86c1-44db-88cd-ce0dc329c882 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d1ae0a54bc0420182194d607a5b85ee, user_id=626e3fff43c949a19327fb1df621ee21, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:10.239855 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-72ba6309-c5ff-4ed2-b290-52b0950caa36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07dc98af8ae14cb49de1695c37d75af3', 'request_id': 'req-72ba6309-c5ff-4ed2-b290-52b0950caa36'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:10.240377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a69f4b2-86c1-44db-88cd-ce0dc329c882 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:10.247610 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-72ba6309-c5ff-4ed2-b290-52b0950caa36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07dc98af8ae14cb49de1695c37d75af3', 'request_id': 'req-72ba6309-c5ff-4ed2-b290-52b0950caa36'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:10.256887 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-72ba6309-c5ff-4ed2-b290-52b0950caa36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07dc98af8ae14cb49de1695c37d75af3', 'request_id': 'req-72ba6309-c5ff-4ed2-b290-52b0950caa36'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:10.257480 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 913/1806] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:10 2022] PUT /identity/v3/projects/7d1ae0a54bc0420182194d607a5b85ee/users/626e3fff43c949a19327fb1df621ee21/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:33:10.261419 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69f4b2-86c1-44db-88cd-ce0dc329c882 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:10.277817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-72ba6309-c5ff-4ed2-b290-52b0950caa36 admin admin] The token cache is being invalidate because project 07dc98af8ae14cb49de1695c37d75af3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:33:10.278343 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-72ba6309-c5ff-4ed2-b290-52b0950caa36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:10.278868 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-72ba6309-c5ff-4ed2-b290-52b0950caa36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:10.283531 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 894/1807] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:33:10 2022] DELETE /identity/v3/projects/07dc98af8ae14cb49de1695c37d75af3 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:10.284384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba6309-c5ff-4ed2-b290-52b0950caa36 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:10.287393 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-72ba6309-c5ff-4ed2-b290-52b0950caa36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:10.337303 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69f4b2-86c1-44db-88cd-ce0dc329c882 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:10.342710 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4444e2b-ea25-4c09-94b3-a05455f88262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:10.343332 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4444e2b-ea25-4c09-94b3-a05455f88262 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:10.343615 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4444e2b-ea25-4c09-94b3-a05455f88262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:10.343893 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4444e2b-ea25-4c09-94b3-a05455f88262 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:10.344662 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4444e2b-ea25-4c09-94b3-a05455f88262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:10.346309 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4444e2b-ea25-4c09-94b3-a05455f88262 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:10.361069 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 914/1808] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:10.365598 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba6309-c5ff-4ed2-b290-52b0950caa36 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:10.380833 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4444e2b-ea25-4c09-94b3-a05455f88262 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:10.381981 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48140cb-3cd8-485b-947f-cdbc60323430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:10.382901 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-e4444e2b-ea25-4c09-94b3-a05455f88262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:10.383084 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48140cb-3cd8-485b-947f-cdbc60323430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:10.383511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48140cb-3cd8-485b-947f-cdbc60323430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:10.383929 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48140cb-3cd8-485b-947f-cdbc60323430 admin admin] PATH_INFO: `/v3/users/38167209ee354f6cafca82f353948b94` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:10.385530 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48140cb-3cd8-485b-947f-cdbc60323430 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38167209ee354f6cafca82f353948b94)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:10.387639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48140cb-3cd8-485b-947f-cdbc60323430 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:10.392024 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:10.397545 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4444e2b-ea25-4c09-94b3-a05455f88262 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:10.404466 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982288b-f78b-49dd-8e60-d85cfa0151c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:10.405217 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982288b-f78b-49dd-8e60-d85cfa0151c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:10.405569 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982288b-f78b-49dd-8e60-d85cfa0151c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:10.405961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982288b-f78b-49dd-8e60-d85cfa0151c5 admin admin] PATH_INFO: `/v3/projects/7d1ae0a54bc0420182194d607a5b85ee/users/626e3fff43c949a19327fb1df621ee21/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:10.408267 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e982288b-f78b-49dd-8e60-d85cfa0151c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d1ae0a54bc0420182194d607a5b85ee, user_id=626e3fff43c949a19327fb1df621ee21, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:10.413384 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e982288b-f78b-49dd-8e60-d85cfa0151c5 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:10.424218 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 915/1809] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:10 2022] PUT /identity/v3/projects/7d1ae0a54bc0420182194d607a5b85ee/users/626e3fff43c949a19327fb1df621ee21/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:33:10.436373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e34f7-5cbb-4b1d-9c89-c3a247630fcf None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:10.436791 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e34f7-5cbb-4b1d-9c89-c3a247630fcf None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:10.437147 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e34f7-5cbb-4b1d-9c89-c3a247630fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:10.457767 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-d48140cb-3cd8-485b-947f-cdbc60323430 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38167209ee354f6cafca82f353948b94'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:10.458601 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-d48140cb-3cd8-485b-947f-cdbc60323430 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38167209ee354f6cafca82f353948b94'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:10.459252 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-d48140cb-3cd8-485b-947f-cdbc60323430 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38167209ee354f6cafca82f353948b94'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:10.468892 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-d48140cb-3cd8-485b-947f-cdbc60323430 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38167209ee354f6cafca82f353948b94'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:10.472567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-9b5e34f7-5cbb-4b1d-9c89-c3a247630fcf None None] MFA Rules not processed for user `626e3fff43c949a19327fb1df621ee21`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:10.474347 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-9b5e34f7-5cbb-4b1d-9c89-c3a247630fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:10.482355 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-d48140cb-3cd8-485b-947f-cdbc60323430 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38167209ee354f6cafca82f353948b94'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:10.504613 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 895/1810] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:33:10 2022] DELETE /identity/v3/users/38167209ee354f6cafca82f353948b94 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:10.565976 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 916/1811] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:10 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:10.578694 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48140cb-3cd8-485b-947f-cdbc60323430 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:10.580123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-d48140cb-3cd8-485b-947f-cdbc60323430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:10.613936 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:10.667341 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48140cb-3cd8-485b-947f-cdbc60323430 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:10.672767 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e326fa-e4bb-47bd-b449-05fa568c6c46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:10.677906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e326fa-e4bb-47bd-b449-05fa568c6c46 service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:10.678550 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e326fa-e4bb-47bd-b449-05fa568c6c46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:10.678931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e326fa-e4bb-47bd-b449-05fa568c6c46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:10.804070 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e326fa-e4bb-47bd-b449-05fa568c6c46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:10.805847 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e326fa-e4bb-47bd-b449-05fa568c6c46 service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:10.819687 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 896/1812] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:10 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:33:10.914737 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e34f7-5cbb-4b1d-9c89-c3a247630fcf None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:11.008225 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e34f7-5cbb-4b1d-9c89-c3a247630fcf None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:11.016587 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe8746-6a52-4159-9dd4-25953b24652f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:11.017507 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe8746-6a52-4159-9dd4-25953b24652f service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:11.018069 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe8746-6a52-4159-9dd4-25953b24652f service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:11.018818 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe8746-6a52-4159-9dd4-25953b24652f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:11.031624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-76fe8746-6a52-4159-9dd4-25953b24652f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:11.124853 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fe8746-6a52-4159-9dd4-25953b24652f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:11.126839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fe8746-6a52-4159-9dd4-25953b24652f service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:11.138640 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 917/1813] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:33:11.278041 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d804c-480e-403e-8c30-ac1ae1845587 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:11.281458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d804c-480e-403e-8c30-ac1ae1845587 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:11.281848 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d804c-480e-403e-8c30-ac1ae1845587 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:11.286166 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 897/1814] 10.222.0.49 () {54 vars in 921 bytes} [Mon Sep 26 10:33:11 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:33:11.304133 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15690d9e-f86f-469c-aa0c-789ce13d37c8 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:11.304605 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15690d9e-f86f-469c-aa0c-789ce13d37c8 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:11.305109 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15690d9e-f86f-469c-aa0c-789ce13d37c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:11.342798 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-15690d9e-f86f-469c-aa0c-789ce13d37c8 None None] MFA Rules not processed for user `cee41014d5d043888c6b399710bccab4`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:11.344250 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-15690d9e-f86f-469c-aa0c-789ce13d37c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:11.347927 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d804c-480e-403e-8c30-ac1ae1845587 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:11.350547 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-582d804c-480e-403e-8c30-ac1ae1845587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:11.373957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:11.435370 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 918/1815] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Sep 26 10:33:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:11.437955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d804c-480e-403e-8c30-ac1ae1845587 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:11.442576 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f7cba-22ed-4a34-8ca9-c3c1a1feddfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:11.443215 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f7cba-22ed-4a34-8ca9-c3c1a1feddfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:11.443491 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f7cba-22ed-4a34-8ca9-c3c1a1feddfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:11.443764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f7cba-22ed-4a34-8ca9-c3c1a1feddfa admin admin] PATH_INFO: `/v3/projects/f9728e815fe04468803df2c9579ba756` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:11.444877 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f7cba-22ed-4a34-8ca9-c3c1a1feddfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9728e815fe04468803df2c9579ba756)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:11.446825 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f7cba-22ed-4a34-8ca9-c3c1a1feddfa admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:11.460994 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15690d9e-f86f-469c-aa0c-789ce13d37c8 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:11.469608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-15690d9e-f86f-469c-aa0c-789ce13d37c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:11.493848 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15690d9e-f86f-469c-aa0c-789ce13d37c8 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:11.502483 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e04f7cba-22ed-4a34-8ca9-c3c1a1feddfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9728e815fe04468803df2c9579ba756', 'request_id': 'req-e04f7cba-22ed-4a34-8ca9-c3c1a1feddfa'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:11.510762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e04f7cba-22ed-4a34-8ca9-c3c1a1feddfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9728e815fe04468803df2c9579ba756', 'request_id': 'req-e04f7cba-22ed-4a34-8ca9-c3c1a1feddfa'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:11.518236 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e04f7cba-22ed-4a34-8ca9-c3c1a1feddfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9728e815fe04468803df2c9579ba756', 'request_id': 'req-e04f7cba-22ed-4a34-8ca9-c3c1a1feddfa'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:11.538531 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e04f7cba-22ed-4a34-8ca9-c3c1a1feddfa admin admin] The token cache is being invalidate because project f9728e815fe04468803df2c9579ba756 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:33:11.538824 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e04f7cba-22ed-4a34-8ca9-c3c1a1feddfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:11.539535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e04f7cba-22ed-4a34-8ca9-c3c1a1feddfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:11.541615 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 898/1816] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:33:11 2022] DELETE /identity/v3/projects/f9728e815fe04468803df2c9579ba756 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:33:11.547276 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff2569-4485-4035-afe8-39a1a5daa999 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:11.547754 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff2569-4485-4035-afe8-39a1a5daa999 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:11.547880 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff2569-4485-4035-afe8-39a1a5daa999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:11.557994 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:11.571647 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-a7ff2569-4485-4035-afe8-39a1a5daa999 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:11.573700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-a7ff2569-4485-4035-afe8-39a1a5daa999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:11.587218 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b71495-2557-4d81-a82f-1b8851431d56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:11.587820 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b71495-2557-4d81-a82f-1b8851431d56 service nova] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:11.588125 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b71495-2557-4d81-a82f-1b8851431d56 service nova] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:11.588405 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b71495-2557-4d81-a82f-1b8851431d56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:11.590165 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-d6b71495-2557-4d81-a82f-1b8851431d56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:11.603038 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b71495-2557-4d81-a82f-1b8851431d56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:11.604684 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b71495-2557-4d81-a82f-1b8851431d56 service nova] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:11.615584 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 919/1817] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:33:11.642071 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 899/1818] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:11.650651 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c51e37-9802-4de0-9d16-8e955a939c4c None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:11.650946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c51e37-9802-4de0-9d16-8e955a939c4c None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:11.651288 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c51e37-9802-4de0-9d16-8e955a939c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:11.671558 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-66c51e37-9802-4de0-9d16-8e955a939c4c None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:11.672810 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-66c51e37-9802-4de0-9d16-8e955a939c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:11.680704 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 920/1819] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:11.685597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff2569-4485-4035-afe8-39a1a5daa999 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:11.695351 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff2569-4485-4035-afe8-39a1a5daa999 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:11.700415 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352bbfe-2592-4d3d-a650-16fab1cec4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:11.701005 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352bbfe-2592-4d3d-a650-16fab1cec4b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:11.701281 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352bbfe-2592-4d3d-a650-16fab1cec4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:11.701552 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352bbfe-2592-4d3d-a650-16fab1cec4b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:11.702248 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352bbfe-2592-4d3d-a650-16fab1cec4b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:33:11.702549 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352bbfe-2592-4d3d-a650-16fab1cec4b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:11.704139 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352bbfe-2592-4d3d-a650-16fab1cec4b7 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:11.715503 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 900/1820] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:33:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:33:11.719536 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c51e37-9802-4de0-9d16-8e955a939c4c None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:11.729003 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c51e37-9802-4de0-9d16-8e955a939c4c None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:11.733615 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977a835-d5d6-4a16-b471-efdae7e709f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:11.734234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977a835-d5d6-4a16-b471-efdae7e709f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:11.734517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977a835-d5d6-4a16-b471-efdae7e709f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:11.734788 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977a835-d5d6-4a16-b471-efdae7e709f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:11.735424 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4977a835-d5d6-4a16-b471-efdae7e709f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:11.737294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4977a835-d5d6-4a16-b471-efdae7e709f2 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:11.764513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 921/1821] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:33:11 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:11.768676 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352bbfe-2592-4d3d-a650-16fab1cec4b7 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:11.779379 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352bbfe-2592-4d3d-a650-16fab1cec4b7 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:11.844086 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4557c-066b-400d-91ca-215788881afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:11.844696 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4557c-066b-400d-91ca-215788881afb admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:11.844978 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4557c-066b-400d-91ca-215788881afb admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:11.845252 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4557c-066b-400d-91ca-215788881afb admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:11.845895 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df4557c-066b-400d-91ca-215788881afb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:11.847843 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df4557c-066b-400d-91ca-215788881afb admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:11.875695 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 901/1822] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:33:11 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:11.880436 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977a835-d5d6-4a16-b471-efdae7e709f2 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:11.891988 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977a835-d5d6-4a16-b471-efdae7e709f2 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:11.897167 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cd4f2-de2f-4365-954e-65cc858a2394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:11.897852 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cd4f2-de2f-4365-954e-65cc858a2394 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:11.898270 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cd4f2-de2f-4365-954e-65cc858a2394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:11.898445 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cd4f2-de2f-4365-954e-65cc858a2394 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:11.899076 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8cd4f2-de2f-4365-954e-65cc858a2394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:11.901236 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8cd4f2-de2f-4365-954e-65cc858a2394 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:11.912105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 922/1823] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:11.915786 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4557c-066b-400d-91ca-215788881afb admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:11.927481 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4557c-066b-400d-91ca-215788881afb admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:11.935323 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca74ea-12fc-4eea-9397-5eaaac37837c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:11.936008 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca74ea-12fc-4eea-9397-5eaaac37837c admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:11.936354 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca74ea-12fc-4eea-9397-5eaaac37837c admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:11.936689 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca74ea-12fc-4eea-9397-5eaaac37837c admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:11.937417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beca74ea-12fc-4eea-9397-5eaaac37837c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:11.939100 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beca74ea-12fc-4eea-9397-5eaaac37837c admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:11.950512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 902/1824] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:11.954662 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cd4f2-de2f-4365-954e-65cc858a2394 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:11.968516 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cd4f2-de2f-4365-954e-65cc858a2394 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:11.973436 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb1a7c-df73-49b5-a44e-61b609b3850f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:11.974113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb1a7c-df73-49b5-a44e-61b609b3850f admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:11.974393 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb1a7c-df73-49b5-a44e-61b609b3850f admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:11.974665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb1a7c-df73-49b5-a44e-61b609b3850f admin admin] PATH_INFO: `/v3/projects/a866fb8dd4c04d478a7b5ddb59b018b0/users/643dc4ec13ca4043a8bc9fc310b3ccbe/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:11.988988 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb1a7c-df73-49b5-a44e-61b609b3850f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a866fb8dd4c04d478a7b5ddb59b018b0, user_id=643dc4ec13ca4043a8bc9fc310b3ccbe, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:11.991660 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb1a7c-df73-49b5-a44e-61b609b3850f admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:12.004406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 923/1825] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:11 2022] PUT /identity/v3/projects/a866fb8dd4c04d478a7b5ddb59b018b0/users/643dc4ec13ca4043a8bc9fc310b3ccbe/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:12.015158 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42494f7-ce53-46ca-b767-0925b29992fd None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:12.015383 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42494f7-ce53-46ca-b767-0925b29992fd None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:12.015654 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42494f7-ce53-46ca-b767-0925b29992fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:12.044626 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-a42494f7-ce53-46ca-b767-0925b29992fd None None] MFA Rules not processed for user `643dc4ec13ca4043a8bc9fc310b3ccbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:12.046151 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-a42494f7-ce53-46ca-b767-0925b29992fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:12.110673 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:12.114672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 903/1826] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:12.126341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb1a7c-df73-49b5-a44e-61b609b3850f admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:12.128057 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-5fbb1a7c-df73-49b5-a44e-61b609b3850f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:12.148281 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42494f7-ce53-46ca-b767-0925b29992fd None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:12.149670 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-a42494f7-ce53-46ca-b767-0925b29992fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:12.182889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:12.230912 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb1a7c-df73-49b5-a44e-61b609b3850f admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:12.232197 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42494f7-ce53-46ca-b767-0925b29992fd None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:12.235513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2653c-b785-4892-9365-9847c08c1a7a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41b242fbe2504f8cac26ce4ff46f5390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:12.236213 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2653c-b785-4892-9365-9847c08c1a7a service glance] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:12.236330 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754559b-8703-4a2e-a570-9ebe8784a4cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:12.236679 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2653c-b785-4892-9365-9847c08c1a7a service glance] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:12.237153 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2653c-b785-4892-9365-9847c08c1a7a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:12.237566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754559b-8703-4a2e-a570-9ebe8784a4cc service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:12.237860 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754559b-8703-4a2e-a570-9ebe8784a4cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:12.238181 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754559b-8703-4a2e-a570-9ebe8784a4cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:12.239998 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-b754559b-8703-4a2e-a570-9ebe8784a4cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:12.247470 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a2653c-b785-4892-9365-9847c08c1a7a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:12.248481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a2653c-b785-4892-9365-9847c08c1a7a service glance] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:12.258488 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 924/1827] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:33:12.331652 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754559b-8703-4a2e-a570-9ebe8784a4cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:12.333078 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754559b-8703-4a2e-a570-9ebe8784a4cc service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:12.343840 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 904/1828] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:33:15.320148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2653c-b785-4892-9365-9847c08c1a7a service glance] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:15.321828 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-f3a2653c-b785-4892-9365-9847c08c1a7a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:15.409570 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2653c-b785-4892-9365-9847c08c1a7a service glance] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:15.418601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00a9d2-eb2f-4d03-b272-2439b62d5b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:15.419262 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00a9d2-eb2f-4d03-b272-2439b62d5b96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:15.419548 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00a9d2-eb2f-4d03-b272-2439b62d5b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:15.419819 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00a9d2-eb2f-4d03-b272-2439b62d5b96 admin admin] PATH_INFO: `/v3/users/28e32d81b52c4e71bdae84b3e68fbecd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:15.421082 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a00a9d2-eb2f-4d03-b272-2439b62d5b96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28e32d81b52c4e71bdae84b3e68fbecd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:15.423549 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a00a9d2-eb2f-4d03-b272-2439b62d5b96 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:15.492333 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-1a00a9d2-eb2f-4d03-b272-2439b62d5b96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28e32d81b52c4e71bdae84b3e68fbecd'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:15.493111 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-1a00a9d2-eb2f-4d03-b272-2439b62d5b96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28e32d81b52c4e71bdae84b3e68fbecd'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:15.504680 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-1a00a9d2-eb2f-4d03-b272-2439b62d5b96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28e32d81b52c4e71bdae84b3e68fbecd'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:15.532529 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-1a00a9d2-eb2f-4d03-b272-2439b62d5b96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28e32d81b52c4e71bdae84b3e68fbecd'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:15.533289 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-1a00a9d2-eb2f-4d03-b272-2439b62d5b96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28e32d81b52c4e71bdae84b3e68fbecd'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:15.550817 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 925/1829] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:33:15 2022] DELETE /identity/v3/users/28e32d81b52c4e71bdae84b3e68fbecd => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:15.555315 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ca371-3c2e-40d4-bec4-c72ccbfe52f4 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:15.555789 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ca371-3c2e-40d4-bec4-c72ccbfe52f4 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:15.555864 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ca371-3c2e-40d4-bec4-c72ccbfe52f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:15.576989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-3e2ca371-3c2e-40d4-bec4-c72ccbfe52f4 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:15.581680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-3e2ca371-3c2e-40d4-bec4-c72ccbfe52f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:15.659095 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 905/1830] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:33:15.670281 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00a9d2-eb2f-4d03-b272-2439b62d5b96 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:15.671858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-1a00a9d2-eb2f-4d03-b272-2439b62d5b96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:15.720904 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:15.736395 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00a9d2-eb2f-4d03-b272-2439b62d5b96 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:15.740553 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3f401-9dee-407a-a736-dda08c71f0f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:15.741203 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3f401-9dee-407a-a736-dda08c71f0f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:15.741486 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3f401-9dee-407a-a736-dda08c71f0f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:15.741760 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3f401-9dee-407a-a736-dda08c71f0f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:15.752232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab3f401-9dee-407a-a736-dda08c71f0f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:15.753596 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab3f401-9dee-407a-a736-dda08c71f0f9 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:15.763207 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 926/1831] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:33:16.081620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ca371-3c2e-40d4-bec4-c72ccbfe52f4 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:16.083055 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-3e2ca371-3c2e-40d4-bec4-c72ccbfe52f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:16.096273 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ca371-3c2e-40d4-bec4-c72ccbfe52f4 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:16.101210 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48861d2e-0098-4807-ad41-1ca887f30033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:16.101841 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48861d2e-0098-4807-ad41-1ca887f30033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:16.102151 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48861d2e-0098-4807-ad41-1ca887f30033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:16.102459 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48861d2e-0098-4807-ad41-1ca887f30033 admin admin] PATH_INFO: `/v3/projects/1cdadf27e69143d1b4a215c15f9c57b2` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:16.103585 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48861d2e-0098-4807-ad41-1ca887f30033 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cdadf27e69143d1b4a215c15f9c57b2)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:16.106126 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48861d2e-0098-4807-ad41-1ca887f30033 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:16.154472 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:16.157489 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-48861d2e-0098-4807-ad41-1ca887f30033 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cdadf27e69143d1b4a215c15f9c57b2', 'request_id': 'req-48861d2e-0098-4807-ad41-1ca887f30033'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:16.164349 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-48861d2e-0098-4807-ad41-1ca887f30033 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cdadf27e69143d1b4a215c15f9c57b2', 'request_id': 'req-48861d2e-0098-4807-ad41-1ca887f30033'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:16.172675 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-48861d2e-0098-4807-ad41-1ca887f30033 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cdadf27e69143d1b4a215c15f9c57b2', 'request_id': 'req-48861d2e-0098-4807-ad41-1ca887f30033'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:16.190376 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-48861d2e-0098-4807-ad41-1ca887f30033 admin admin] The token cache is being invalidate because project 1cdadf27e69143d1b4a215c15f9c57b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:33:16.190510 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-48861d2e-0098-4807-ad41-1ca887f30033 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:16.191138 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-48861d2e-0098-4807-ad41-1ca887f30033 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:16.196042 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 906/1832] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:33:16 2022] DELETE /identity/v3/projects/1cdadf27e69143d1b4a215c15f9c57b2 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:16.198615 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3f401-9dee-407a-a736-dda08c71f0f9 service neutron] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:16.209398 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3f401-9dee-407a-a736-dda08c71f0f9 service neutron] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:16.273525 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9451c1-7041-480e-9c72-3e409c9475a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:16.274109 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9451c1-7041-480e-9c72-3e409c9475a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:16.274391 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9451c1-7041-480e-9c72-3e409c9475a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:16.274665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9451c1-7041-480e-9c72-3e409c9475a2 admin admin] PATH_INFO: `/v3/users/626e3fff43c949a19327fb1df621ee21` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:16.275792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9451c1-7041-480e-9c72-3e409c9475a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=626e3fff43c949a19327fb1df621ee21)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:16.278255 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9451c1-7041-480e-9c72-3e409c9475a2 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:16.327666 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-3e9451c1-7041-480e-9c72-3e409c9475a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '626e3fff43c949a19327fb1df621ee21'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:16.328434 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-3e9451c1-7041-480e-9c72-3e409c9475a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '626e3fff43c949a19327fb1df621ee21'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:16.335898 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-3e9451c1-7041-480e-9c72-3e409c9475a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '626e3fff43c949a19327fb1df621ee21'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:16.351582 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-3e9451c1-7041-480e-9c72-3e409c9475a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '626e3fff43c949a19327fb1df621ee21'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:16.352495 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-3e9451c1-7041-480e-9c72-3e409c9475a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '626e3fff43c949a19327fb1df621ee21'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:16.366079 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 927/1833] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:33:16 2022] DELETE /identity/v3/users/626e3fff43c949a19327fb1df621ee21 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:16.720105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48861d2e-0098-4807-ad41-1ca887f30033 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:16.721681 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-48861d2e-0098-4807-ad41-1ca887f30033 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:16.735027 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48861d2e-0098-4807-ad41-1ca887f30033 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:16.814526 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9bdfa-5cad-43bb-a943-91c9c4d58300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:16.815163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9bdfa-5cad-43bb-a943-91c9c4d58300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:16.815440 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9bdfa-5cad-43bb-a943-91c9c4d58300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:16.815708 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9bdfa-5cad-43bb-a943-91c9c4d58300 admin admin] PATH_INFO: `/v3/projects/7d1ae0a54bc0420182194d607a5b85ee` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:16.816808 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf9bdfa-5cad-43bb-a943-91c9c4d58300 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d1ae0a54bc0420182194d607a5b85ee)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:16.818785 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf9bdfa-5cad-43bb-a943-91c9c4d58300 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:16.866445 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-caf9bdfa-5cad-43bb-a943-91c9c4d58300 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d1ae0a54bc0420182194d607a5b85ee', 'request_id': 'req-caf9bdfa-5cad-43bb-a943-91c9c4d58300'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:16.873101 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-caf9bdfa-5cad-43bb-a943-91c9c4d58300 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d1ae0a54bc0420182194d607a5b85ee', 'request_id': 'req-caf9bdfa-5cad-43bb-a943-91c9c4d58300'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:16.878700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-caf9bdfa-5cad-43bb-a943-91c9c4d58300 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d1ae0a54bc0420182194d607a5b85ee', 'request_id': 'req-caf9bdfa-5cad-43bb-a943-91c9c4d58300'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:16.892142 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-caf9bdfa-5cad-43bb-a943-91c9c4d58300 admin admin] The token cache is being invalidate because project 7d1ae0a54bc0420182194d607a5b85ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:33:16.892432 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-caf9bdfa-5cad-43bb-a943-91c9c4d58300 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:16.893055 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-caf9bdfa-5cad-43bb-a943-91c9c4d58300 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:16.894583 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 907/1834] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:33:16 2022] DELETE /identity/v3/projects/7d1ae0a54bc0420182194d607a5b85ee => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:16.900361 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e5005-9c21-4f0d-93f3-4cbc2e8c418a None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:16.900649 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e5005-9c21-4f0d-93f3-4cbc2e8c418a None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:16.900925 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e5005-9c21-4f0d-93f3-4cbc2e8c418a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:16.922044 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-177e5005-9c21-4f0d-93f3-4cbc2e8c418a None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:16.923352 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-177e5005-9c21-4f0d-93f3-4cbc2e8c418a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:16.983701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 928/1835] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:16.991496 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc31c8-5263-4ada-9e9f-730d3d1cf3ae None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:16.991726 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc31c8-5263-4ada-9e9f-730d3d1cf3ae None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:16.992002 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc31c8-5263-4ada-9e9f-730d3d1cf3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:17.013689 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-ccfc31c8-5263-4ada-9e9f-730d3d1cf3ae None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:17.014963 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-ccfc31c8-5263-4ada-9e9f-730d3d1cf3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:17.022988 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 908/1836] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:17.028843 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e5005-9c21-4f0d-93f3-4cbc2e8c418a None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:17.038678 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e5005-9c21-4f0d-93f3-4cbc2e8c418a None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:17.043208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed005981-f658-440d-81fb-81f12e2fbabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:17.043946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed005981-f658-440d-81fb-81f12e2fbabb admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:17.044227 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed005981-f658-440d-81fb-81f12e2fbabb admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:17.044509 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed005981-f658-440d-81fb-81f12e2fbabb admin admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:17.045145 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed005981-f658-440d-81fb-81f12e2fbabb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:33:17.045572 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed005981-f658-440d-81fb-81f12e2fbabb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:17.047196 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed005981-f658-440d-81fb-81f12e2fbabb admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:17.049871 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:17.069972 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 929/1837] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:33:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:33:17.071182 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfc31c8-5263-4ada-9e9f-730d3d1cf3ae None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:17.081828 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfc31c8-5263-4ada-9e9f-730d3d1cf3ae None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:17.086732 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d2678-ae1a-4352-a5f7-b677d71b212f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:17.087296 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d2678-ae1a-4352-a5f7-b677d71b212f admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:17.096744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d2678-ae1a-4352-a5f7-b677d71b212f admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:17.097171 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d2678-ae1a-4352-a5f7-b677d71b212f admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:17.097879 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d2678-ae1a-4352-a5f7-b677d71b212f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:17.100160 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d2678-ae1a-4352-a5f7-b677d71b212f admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:17.134902 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 909/1838] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:33:17 2022] POST /identity/v3/projects => generated 393 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:17.139740 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed005981-f658-440d-81fb-81f12e2fbabb admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:17.149730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed005981-f658-440d-81fb-81f12e2fbabb admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:17.209088 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba2fac-6594-4f73-808c-dcd5795b7fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:17.209708 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba2fac-6594-4f73-808c-dcd5795b7fe8 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:17.209986 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba2fac-6594-4f73-808c-dcd5795b7fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:17.210282 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba2fac-6594-4f73-808c-dcd5795b7fe8 admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:17.210932 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fba2fac-6594-4f73-808c-dcd5795b7fe8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:17.212931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fba2fac-6594-4f73-808c-dcd5795b7fe8 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:17.234948 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 930/1839] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:33:17 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:17.239190 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d2678-ae1a-4352-a5f7-b677d71b212f admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:17.250979 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d2678-ae1a-4352-a5f7-b677d71b212f admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:17.255756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd3eb2-4679-429e-b3bf-d570fbfda577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:17.256392 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd3eb2-4679-429e-b3bf-d570fbfda577 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:17.256671 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd3eb2-4679-429e-b3bf-d570fbfda577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:17.256969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd3eb2-4679-429e-b3bf-d570fbfda577 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:17.257612 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd3eb2-4679-429e-b3bf-d570fbfda577 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:17.259312 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd3eb2-4679-429e-b3bf-d570fbfda577 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:17.269403 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 910/1840] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:33:17.273250 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba2fac-6594-4f73-808c-dcd5795b7fe8 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:17.283080 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba2fac-6594-4f73-808c-dcd5795b7fe8 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:17.287588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ca52b-aad9-404d-bca5-753966de12a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:17.288176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ca52b-aad9-404d-bca5-753966de12a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:17.288457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ca52b-aad9-404d-bca5-753966de12a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:17.288776 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ca52b-aad9-404d-bca5-753966de12a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:17.289368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ca52b-aad9-404d-bca5-753966de12a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:17.291088 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ca52b-aad9-404d-bca5-753966de12a3 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:17.303229 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 931/1841] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:17.307588 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd3eb2-4679-429e-b3bf-d570fbfda577 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:17.318080 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd3eb2-4679-429e-b3bf-d570fbfda577 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:17.322657 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156fd8f-efae-41d5-8512-7c879d4991ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:17.323413 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156fd8f-efae-41d5-8512-7c879d4991ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:17.323695 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156fd8f-efae-41d5-8512-7c879d4991ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:17.323969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156fd8f-efae-41d5-8512-7c879d4991ae admin admin] PATH_INFO: `/v3/projects/544efae6a97d481693492e590c725f3a/users/ff6de7ec6d2f4f1b95b199caf3b7deac/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:17.338190 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e156fd8f-efae-41d5-8512-7c879d4991ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=544efae6a97d481693492e590c725f3a, user_id=ff6de7ec6d2f4f1b95b199caf3b7deac, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:17.340850 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e156fd8f-efae-41d5-8512-7c879d4991ae admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:17.350183 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:17.355839 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 911/1842] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:17 2022] PUT /identity/v3/projects/544efae6a97d481693492e590c725f3a/users/ff6de7ec6d2f4f1b95b199caf3b7deac/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:33:17.367754 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a57d0-5b1c-4527-8f55-fe60f57ae132 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:17.368239 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a57d0-5b1c-4527-8f55-fe60f57ae132 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:17.368501 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a57d0-5b1c-4527-8f55-fe60f57ae132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:17.398763 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-6e3a57d0-5b1c-4527-8f55-fe60f57ae132 None None] MFA Rules not processed for user `ff6de7ec6d2f4f1b95b199caf3b7deac`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:17.399998 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-6e3a57d0-5b1c-4527-8f55-fe60f57ae132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:17.471121 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 932/1843] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:17.475906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156fd8f-efae-41d5-8512-7c879d4991ae admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:17.547850 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156fd8f-efae-41d5-8512-7c879d4991ae admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:17.553832 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6991fb-579d-49e9-8bfe-9616ec40b83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:17.554471 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6991fb-579d-49e9-8bfe-9616ec40b83c admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:17.554755 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6991fb-579d-49e9-8bfe-9616ec40b83c admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:17.555030 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6991fb-579d-49e9-8bfe-9616ec40b83c admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:17.555676 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6991fb-579d-49e9-8bfe-9616ec40b83c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:17.557693 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6991fb-579d-49e9-8bfe-9616ec40b83c admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:17.584800 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 912/1844] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:33:17 2022] POST /identity/v3/projects => generated 391 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:17.589519 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a57d0-5b1c-4527-8f55-fe60f57ae132 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:17.600673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a57d0-5b1c-4527-8f55-fe60f57ae132 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:17.658470 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c265f20-7997-4040-9722-7aa7e48b209e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:17.659191 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c265f20-7997-4040-9722-7aa7e48b209e admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:17.659538 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c265f20-7997-4040-9722-7aa7e48b209e admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:17.659805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c265f20-7997-4040-9722-7aa7e48b209e admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:17.660436 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c265f20-7997-4040-9722-7aa7e48b209e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:17.662515 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c265f20-7997-4040-9722-7aa7e48b209e admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:17.672024 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:17.685009 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 933/1845] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:33:17 2022] POST /identity/v3/users => generated 302 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:17.689041 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6991fb-579d-49e9-8bfe-9616ec40b83c admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:17.699997 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6991fb-579d-49e9-8bfe-9616ec40b83c admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:17.704546 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebbc91-9d2c-48c9-879a-43eddf7a6214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:17.705145 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebbc91-9d2c-48c9-879a-43eddf7a6214 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:17.705424 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebbc91-9d2c-48c9-879a-43eddf7a6214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:17.705875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebbc91-9d2c-48c9-879a-43eddf7a6214 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:17.706485 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ebbc91-9d2c-48c9-879a-43eddf7a6214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:17.708200 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ebbc91-9d2c-48c9-879a-43eddf7a6214 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:17.719841 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 913/1846] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:17.724329 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c265f20-7997-4040-9722-7aa7e48b209e admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:17.734428 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c265f20-7997-4040-9722-7aa7e48b209e admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:17.739037 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a81e5-b34a-46e4-873e-067b2abf1c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:17.739680 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a81e5-b34a-46e4-873e-067b2abf1c79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:17.739959 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a81e5-b34a-46e4-873e-067b2abf1c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:17.740234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a81e5-b34a-46e4-873e-067b2abf1c79 admin admin] PATH_INFO: `/v3/projects/0e46f5a013a642118888599a9aa37f0e/users/cb065c4dc9934d2298cf1d518297ba74/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:17.755083 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a81e5-b34a-46e4-873e-067b2abf1c79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e46f5a013a642118888599a9aa37f0e, user_id=cb065c4dc9934d2298cf1d518297ba74, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:17.757790 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a81e5-b34a-46e4-873e-067b2abf1c79 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:17.771275 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 934/1847] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:17 2022] PUT /identity/v3/projects/0e46f5a013a642118888599a9aa37f0e/users/cb065c4dc9934d2298cf1d518297ba74/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:33:17.777999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebbc91-9d2c-48c9-879a-43eddf7a6214 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:17.789742 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebbc91-9d2c-48c9-879a-43eddf7a6214 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:17.856198 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77220274-d63e-4390-aba8-4244589bd9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:17.856814 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77220274-d63e-4390-aba8-4244589bd9bc admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:17.857094 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77220274-d63e-4390-aba8-4244589bd9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:17.857371 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77220274-d63e-4390-aba8-4244589bd9bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:17.858170 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77220274-d63e-4390-aba8-4244589bd9bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:17.859804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77220274-d63e-4390-aba8-4244589bd9bc admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:17.870393 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 914/1848] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:17.874251 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a81e5-b34a-46e4-873e-067b2abf1c79 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:17.885567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a81e5-b34a-46e4-873e-067b2abf1c79 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:17.905008 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff85277-4e12-4375-bc62-0d5ed2f8fcc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:17.905730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff85277-4e12-4375-bc62-0d5ed2f8fcc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:17.906149 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff85277-4e12-4375-bc62-0d5ed2f8fcc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:17.906393 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff85277-4e12-4375-bc62-0d5ed2f8fcc9 admin admin] PATH_INFO: `/v3/projects/0e46f5a013a642118888599a9aa37f0e/users/cb065c4dc9934d2298cf1d518297ba74/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:17.908337 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff85277-4e12-4375-bc62-0d5ed2f8fcc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e46f5a013a642118888599a9aa37f0e, user_id=cb065c4dc9934d2298cf1d518297ba74, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:17.911074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff85277-4e12-4375-bc62-0d5ed2f8fcc9 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:17.922194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 935/1849] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:17 2022] PUT /identity/v3/projects/0e46f5a013a642118888599a9aa37f0e/users/cb065c4dc9934d2298cf1d518297ba74/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:33:17.933373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75569fe9-e8c4-4264-b37d-d2cd734992dd None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:17.934306 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75569fe9-e8c4-4264-b37d-d2cd734992dd None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:17.934306 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75569fe9-e8c4-4264-b37d-d2cd734992dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:17.969140 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-75569fe9-e8c4-4264-b37d-d2cd734992dd None None] MFA Rules not processed for user `cb065c4dc9934d2298cf1d518297ba74`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:17.970595 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-75569fe9-e8c4-4264-b37d-d2cd734992dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:18.041455 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:18.052361 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 915/1850] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:17 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:18.063684 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff85277-4e12-4375-bc62-0d5ed2f8fcc9 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:18.065377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-8ff85277-4e12-4375-bc62-0d5ed2f8fcc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:18.127230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff85277-4e12-4375-bc62-0d5ed2f8fcc9 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:18.130841 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf87c5-4710-45d2-b9a7-d0db62886537 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:18.131479 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf87c5-4710-45d2-b9a7-d0db62886537 service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:18.131756 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf87c5-4710-45d2-b9a7-d0db62886537 service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:18.132032 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf87c5-4710-45d2-b9a7-d0db62886537 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:18.141566 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf87c5-4710-45d2-b9a7-d0db62886537 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:18.143044 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf87c5-4710-45d2-b9a7-d0db62886537 service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:18.152611 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 936/1851] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:18 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:33:18.208269 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75569fe9-e8c4-4264-b37d-d2cd734992dd None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:18.209669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-75569fe9-e8c4-4264-b37d-d2cd734992dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:18.221540 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75569fe9-e8c4-4264-b37d-d2cd734992dd None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:18.225430 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386d3b6-a1f4-435d-9d92-12da6b896c3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:18.226047 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386d3b6-a1f4-435d-9d92-12da6b896c3b service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:18.226334 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386d3b6-a1f4-435d-9d92-12da6b896c3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:18.226607 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386d3b6-a1f4-435d-9d92-12da6b896c3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:18.293835 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0386d3b6-a1f4-435d-9d92-12da6b896c3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:18.295428 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0386d3b6-a1f4-435d-9d92-12da6b896c3b service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:18.305197 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 916/1852] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:18 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:33:26.403819 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70932112-8faa-4e53-8f09-ba51b64d1fd4 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:26.404271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70932112-8faa-4e53-8f09-ba51b64d1fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:26.404861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70932112-8faa-4e53-8f09-ba51b64d1fd4 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:26.413395 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 937/1853] 10.222.0.49 () {54 vars in 921 bytes} [Mon Sep 26 10:33:26 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:33:26.420499 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd96401-cd31-4594-bbdd-974a2a9e372f None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:26.421343 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd96401-cd31-4594-bbdd-974a2a9e372f None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:26.421769 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd96401-cd31-4594-bbdd-974a2a9e372f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:26.458737 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-9bd96401-cd31-4594-bbdd-974a2a9e372f None None] MFA Rules not processed for user `cee41014d5d043888c6b399710bccab4`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:26.460238 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-9bd96401-cd31-4594-bbdd-974a2a9e372f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:26.570709 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 917/1854] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Sep 26 10:33:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:27.041919 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70932112-8faa-4e53-8f09-ba51b64d1fd4 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:27.049440 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-70932112-8faa-4e53-8f09-ba51b64d1fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:27.099687 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70932112-8faa-4e53-8f09-ba51b64d1fd4 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:27.108451 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46c207-d1d6-47e6-9553-efa4c184fcf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:27.109976 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46c207-d1d6-47e6-9553-efa4c184fcf6 service nova] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:27.110298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46c207-d1d6-47e6-9553-efa4c184fcf6 service nova] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:27.110633 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46c207-d1d6-47e6-9553-efa4c184fcf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:27.123421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe46c207-d1d6-47e6-9553-efa4c184fcf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:27.126161 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe46c207-d1d6-47e6-9553-efa4c184fcf6 service nova] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:27.157563 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd96401-cd31-4594-bbdd-974a2a9e372f None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:27.161479 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 938/1855] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:33:27.277011 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd96401-cd31-4594-bbdd-974a2a9e372f None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:27.284647 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f834b-461e-4094-b091-d56a10b6d842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:27.285307 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f834b-461e-4094-b091-d56a10b6d842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:27.285572 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f834b-461e-4094-b091-d56a10b6d842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:27.285844 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f834b-461e-4094-b091-d56a10b6d842 admin admin] PATH_INFO: `/v3/users/ff6de7ec6d2f4f1b95b199caf3b7deac` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:27.287286 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f834b-461e-4094-b091-d56a10b6d842 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff6de7ec6d2f4f1b95b199caf3b7deac)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:27.289029 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f834b-461e-4094-b091-d56a10b6d842 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:27.335508 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:27.364314 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-1f5f834b-461e-4094-b091-d56a10b6d842 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff6de7ec6d2f4f1b95b199caf3b7deac'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:27.365105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-1f5f834b-461e-4094-b091-d56a10b6d842 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff6de7ec6d2f4f1b95b199caf3b7deac'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:27.365828 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-1f5f834b-461e-4094-b091-d56a10b6d842 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff6de7ec6d2f4f1b95b199caf3b7deac'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:27.379316 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-1f5f834b-461e-4094-b091-d56a10b6d842 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff6de7ec6d2f4f1b95b199caf3b7deac'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:27.406584 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-1f5f834b-461e-4094-b091-d56a10b6d842 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff6de7ec6d2f4f1b95b199caf3b7deac'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:27.438459 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 918/1856] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:33:27 2022] DELETE /identity/v3/users/ff6de7ec6d2f4f1b95b199caf3b7deac => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:27.443742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f409197-5aec-449b-b028-580739586066 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:27.444614 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f409197-5aec-449b-b028-580739586066 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:27.445030 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f409197-5aec-449b-b028-580739586066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:27.470699 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-3f409197-5aec-449b-b028-580739586066 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:27.472187 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-3f409197-5aec-449b-b028-580739586066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:27.559392 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 939/1857] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:27.581073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f834b-461e-4094-b091-d56a10b6d842 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:27.584715 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-1f5f834b-461e-4094-b091-d56a10b6d842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:27.707986 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f834b-461e-4094-b091-d56a10b6d842 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:27.711753 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64023467-1318-48c4-a1d0-a25f7725f1e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:27.712365 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64023467-1318-48c4-a1d0-a25f7725f1e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:27.712628 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64023467-1318-48c4-a1d0-a25f7725f1e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:27.712901 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64023467-1318-48c4-a1d0-a25f7725f1e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:27.724309 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64023467-1318-48c4-a1d0-a25f7725f1e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:27.726015 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64023467-1318-48c4-a1d0-a25f7725f1e6 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:27.746569 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 919/1858] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:33:28.076862 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f409197-5aec-449b-b028-580739586066 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:28.078322 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-3f409197-5aec-449b-b028-580739586066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:28.094223 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f409197-5aec-449b-b028-580739586066 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:28.123576 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d056dc-d932-4754-ba7d-696616511d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:28.124494 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d056dc-d932-4754-ba7d-696616511d72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:28.127525 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d056dc-d932-4754-ba7d-696616511d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:28.127978 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d056dc-d932-4754-ba7d-696616511d72 admin admin] PATH_INFO: `/v3/projects/544efae6a97d481693492e590c725f3a` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:28.130466 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d056dc-d932-4754-ba7d-696616511d72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=544efae6a97d481693492e590c725f3a)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:28.133193 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d056dc-d932-4754-ba7d-696616511d72 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:28.191367 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:INSERT=4,keystone:SELECT=93,keystone:DELETE=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:28.218945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f6d056dc-d932-4754-ba7d-696616511d72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '544efae6a97d481693492e590c725f3a', 'request_id': 'req-f6d056dc-d932-4754-ba7d-696616511d72'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:28.230222 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f6d056dc-d932-4754-ba7d-696616511d72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '544efae6a97d481693492e590c725f3a', 'request_id': 'req-f6d056dc-d932-4754-ba7d-696616511d72'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:28.246040 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f6d056dc-d932-4754-ba7d-696616511d72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '544efae6a97d481693492e590c725f3a', 'request_id': 'req-f6d056dc-d932-4754-ba7d-696616511d72'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:28.263537 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f6d056dc-d932-4754-ba7d-696616511d72 admin admin] The token cache is being invalidate because project 544efae6a97d481693492e590c725f3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:33:28.263869 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f6d056dc-d932-4754-ba7d-696616511d72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:28.264574 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f6d056dc-d932-4754-ba7d-696616511d72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:28.267242 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 940/1859] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:33:28 2022] DELETE /identity/v3/projects/544efae6a97d481693492e590c725f3a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:28.270914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64023467-1318-48c4-a1d0-a25f7725f1e6 service neutron] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:28.282490 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64023467-1318-48c4-a1d0-a25f7725f1e6 service neutron] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:28.384630 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed52cd5-a212-446c-8ae3-403f15d921ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:28.385260 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed52cd5-a212-446c-8ae3-403f15d921ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:28.385552 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed52cd5-a212-446c-8ae3-403f15d921ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:28.386391 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed52cd5-a212-446c-8ae3-403f15d921ba admin admin] PATH_INFO: `/v3/users/cb065c4dc9934d2298cf1d518297ba74` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:28.387531 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed52cd5-a212-446c-8ae3-403f15d921ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb065c4dc9934d2298cf1d518297ba74)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:28.389421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed52cd5-a212-446c-8ae3-403f15d921ba admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:28.446449 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-7ed52cd5-a212-446c-8ae3-403f15d921ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb065c4dc9934d2298cf1d518297ba74'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:28.447113 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-7ed52cd5-a212-446c-8ae3-403f15d921ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb065c4dc9934d2298cf1d518297ba74'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:28.447735 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-7ed52cd5-a212-446c-8ae3-403f15d921ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb065c4dc9934d2298cf1d518297ba74'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:28.450364 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:28.456765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-7ed52cd5-a212-446c-8ae3-403f15d921ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb065c4dc9934d2298cf1d518297ba74'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:28.476419 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-7ed52cd5-a212-446c-8ae3-403f15d921ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb065c4dc9934d2298cf1d518297ba74'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:28.491490 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 920/1860] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:33:28 2022] DELETE /identity/v3/users/cb065c4dc9934d2298cf1d518297ba74 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:28.842559 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d056dc-d932-4754-ba7d-696616511d72 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:28.845005 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-f6d056dc-d932-4754-ba7d-696616511d72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:28.882403 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d056dc-d932-4754-ba7d-696616511d72 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:28.963305 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8100c10-347a-4b4a-b971-700ca379ed51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:28.964118 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8100c10-347a-4b4a-b971-700ca379ed51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:28.964379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8100c10-347a-4b4a-b971-700ca379ed51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:28.964723 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8100c10-347a-4b4a-b971-700ca379ed51 admin admin] PATH_INFO: `/v3/projects/0e46f5a013a642118888599a9aa37f0e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:28.965927 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8100c10-347a-4b4a-b971-700ca379ed51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e46f5a013a642118888599a9aa37f0e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:28.967934 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8100c10-347a-4b4a-b971-700ca379ed51 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:29.038235 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d8100c10-347a-4b4a-b971-700ca379ed51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e46f5a013a642118888599a9aa37f0e', 'request_id': 'req-d8100c10-347a-4b4a-b971-700ca379ed51'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:29.046014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d8100c10-347a-4b4a-b971-700ca379ed51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e46f5a013a642118888599a9aa37f0e', 'request_id': 'req-d8100c10-347a-4b4a-b971-700ca379ed51'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:29.058479 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d8100c10-347a-4b4a-b971-700ca379ed51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e46f5a013a642118888599a9aa37f0e', 'request_id': 'req-d8100c10-347a-4b4a-b971-700ca379ed51'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:29.088476 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d8100c10-347a-4b4a-b971-700ca379ed51 admin admin] The token cache is being invalidate because project 0e46f5a013a642118888599a9aa37f0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:33:29.088848 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d8100c10-347a-4b4a-b971-700ca379ed51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:29.090416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d8100c10-347a-4b4a-b971-700ca379ed51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:29.092322 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 941/1861] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:33:28 2022] DELETE /identity/v3/projects/0e46f5a013a642118888599a9aa37f0e => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:29.100401 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0db0f6-7996-4364-9472-8ed77855e262 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:29.100687 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0db0f6-7996-4364-9472-8ed77855e262 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:29.100960 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0db0f6-7996-4364-9472-8ed77855e262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:29.135014 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-0b0db0f6-7996-4364-9472-8ed77855e262 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:29.136823 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-0b0db0f6-7996-4364-9472-8ed77855e262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:29.195767 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 921/1862] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:33:29.203589 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e57b8-f4b6-4725-9a62-5bfc2ae87ee3 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:29.203872 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e57b8-f4b6-4725-9a62-5bfc2ae87ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:29.204148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e57b8-f4b6-4725-9a62-5bfc2ae87ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:29.223890 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-ad9e57b8-f4b6-4725-9a62-5bfc2ae87ee3 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:29.225082 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-ad9e57b8-f4b6-4725-9a62-5bfc2ae87ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:29.233635 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 942/1863] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:29.237833 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0db0f6-7996-4364-9472-8ed77855e262 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:29.248324 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0db0f6-7996-4364-9472-8ed77855e262 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:29.252923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ed7cf-e3da-45c8-b06a-8e1b4a5f2865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:29.253518 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ed7cf-e3da-45c8-b06a-8e1b4a5f2865 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:29.253797 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ed7cf-e3da-45c8-b06a-8e1b4a5f2865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:29.254100 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ed7cf-e3da-45c8-b06a-8e1b4a5f2865 admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:29.254745 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7ed7cf-e3da-45c8-b06a-8e1b4a5f2865 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:33:29.255044 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7ed7cf-e3da-45c8-b06a-8e1b4a5f2865 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:29.256635 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7ed7cf-e3da-45c8-b06a-8e1b4a5f2865 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:29.269047 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 922/1864] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:33:29.273362 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e57b8-f4b6-4725-9a62-5bfc2ae87ee3 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:29.284294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e57b8-f4b6-4725-9a62-5bfc2ae87ee3 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:29.287870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3a6a7-dbb3-4681-9af0-c4cf71522af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:29.288473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3a6a7-dbb3-4681-9af0-c4cf71522af0 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:29.288751 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3a6a7-dbb3-4681-9af0-c4cf71522af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:29.289022 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3a6a7-dbb3-4681-9af0-c4cf71522af0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:29.289667 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd3a6a7-dbb3-4681-9af0-c4cf71522af0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:29.291586 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd3a6a7-dbb3-4681-9af0-c4cf71522af0 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:29.319705 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 943/1865] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:33:29 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:29.325506 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ed7cf-e3da-45c8-b06a-8e1b4a5f2865 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:29.335369 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ed7cf-e3da-45c8-b06a-8e1b4a5f2865 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:29.395541 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b42903d-3845-4520-a3f0-ed57385192c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:29.396148 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b42903d-3845-4520-a3f0-ed57385192c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:29.396480 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b42903d-3845-4520-a3f0-ed57385192c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:29.397248 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b42903d-3845-4520-a3f0-ed57385192c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:29.397358 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b42903d-3845-4520-a3f0-ed57385192c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:29.399256 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b42903d-3845-4520-a3f0-ed57385192c5 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:29.422738 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 923/1866] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:33:29 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:29.426750 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3a6a7-dbb3-4681-9af0-c4cf71522af0 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:29.438183 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3a6a7-dbb3-4681-9af0-c4cf71522af0 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:29.445530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89064a-a7b0-4839-8e35-48844c20819f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:29.446380 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89064a-a7b0-4839-8e35-48844c20819f admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:29.446605 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89064a-a7b0-4839-8e35-48844c20819f admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:29.446873 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89064a-a7b0-4839-8e35-48844c20819f admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:29.447518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae89064a-a7b0-4839-8e35-48844c20819f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:29.449142 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae89064a-a7b0-4839-8e35-48844c20819f admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:29.459817 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 944/1867] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:29.464118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b42903d-3845-4520-a3f0-ed57385192c5 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:29.474644 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b42903d-3845-4520-a3f0-ed57385192c5 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:29.479789 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7330264-3307-4726-a0fc-0e2b110c33f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:29.480573 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7330264-3307-4726-a0fc-0e2b110c33f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:29.481008 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7330264-3307-4726-a0fc-0e2b110c33f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:29.481280 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7330264-3307-4726-a0fc-0e2b110c33f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:29.481954 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7330264-3307-4726-a0fc-0e2b110c33f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:29.483723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7330264-3307-4726-a0fc-0e2b110c33f0 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:29.494856 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 924/1868] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:29.503009 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89064a-a7b0-4839-8e35-48844c20819f admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:29.513087 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89064a-a7b0-4839-8e35-48844c20819f admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:29.517588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a6b70d-0ae6-4ca7-8c17-7088b6eddfc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:29.518262 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6b70d-0ae6-4ca7-8c17-7088b6eddfc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:29.518538 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6b70d-0ae6-4ca7-8c17-7088b6eddfc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:29.518814 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6b70d-0ae6-4ca7-8c17-7088b6eddfc6 admin admin] PATH_INFO: `/v3/projects/72e30732b0864ce0b56fabdc16568d65/users/6b7abedf87624d06acc2260bd6b9838f/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:29.534136 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a6b70d-0ae6-4ca7-8c17-7088b6eddfc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72e30732b0864ce0b56fabdc16568d65, user_id=6b7abedf87624d06acc2260bd6b9838f, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:29.536754 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a6b70d-0ae6-4ca7-8c17-7088b6eddfc6 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:29.539131 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:29.552610 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 945/1869] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:29 2022] PUT /identity/v3/projects/72e30732b0864ce0b56fabdc16568d65/users/6b7abedf87624d06acc2260bd6b9838f/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:29.565303 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c39d789-2f41-4609-bb45-18a3fd46fcce None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:29.565588 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c39d789-2f41-4609-bb45-18a3fd46fcce None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:29.565862 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c39d789-2f41-4609-bb45-18a3fd46fcce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:29.624086 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-7c39d789-2f41-4609-bb45-18a3fd46fcce None None] MFA Rules not processed for user `6b7abedf87624d06acc2260bd6b9838f`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:29.625322 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-7c39d789-2f41-4609-bb45-18a3fd46fcce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:29.647351 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:29.693129 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 925/1870] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:29.697802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a6b70d-0ae6-4ca7-8c17-7088b6eddfc6 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:29.763051 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a6b70d-0ae6-4ca7-8c17-7088b6eddfc6 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:29.767665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb80a7-b4bd-4796-b5e0-6ea78f6d2dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:29.768290 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb80a7-b4bd-4796-b5e0-6ea78f6d2dfe admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:29.768570 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb80a7-b4bd-4796-b5e0-6ea78f6d2dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:29.768845 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb80a7-b4bd-4796-b5e0-6ea78f6d2dfe admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:29.769489 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcb80a7-b4bd-4796-b5e0-6ea78f6d2dfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:29.771408 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcb80a7-b4bd-4796-b5e0-6ea78f6d2dfe admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:29.801970 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 946/1871] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:33:29 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:33:29.803348 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c39d789-2f41-4609-bb45-18a3fd46fcce None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:29.813904 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c39d789-2f41-4609-bb45-18a3fd46fcce None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:29.873454 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56116828-2cfc-43a0-88c1-4c11d0c520e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:29.874101 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56116828-2cfc-43a0-88c1-4c11d0c520e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:29.874376 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56116828-2cfc-43a0-88c1-4c11d0c520e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:29.874647 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56116828-2cfc-43a0-88c1-4c11d0c520e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:29.875257 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56116828-2cfc-43a0-88c1-4c11d0c520e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:29.877140 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56116828-2cfc-43a0-88c1-4c11d0c520e1 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:29.898425 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 926/1872] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:33:29 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:29.903163 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb80a7-b4bd-4796-b5e0-6ea78f6d2dfe admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:29.913469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb80a7-b4bd-4796-b5e0-6ea78f6d2dfe admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:29.919029 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dfe685-3cce-4069-a635-4faabeca54f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:29.919623 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfe685-3cce-4069-a635-4faabeca54f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:29.919904 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfe685-3cce-4069-a635-4faabeca54f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:29.920177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfe685-3cce-4069-a635-4faabeca54f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:29.920816 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dfe685-3cce-4069-a635-4faabeca54f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:29.922454 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dfe685-3cce-4069-a635-4faabeca54f0 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:29.933184 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 947/1873] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:33:29.937191 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56116828-2cfc-43a0-88c1-4c11d0c520e1 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:29.949939 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56116828-2cfc-43a0-88c1-4c11d0c520e1 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:29.955797 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ba2b6-a26b-4a8f-8561-e466398793c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:29.956446 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ba2b6-a26b-4a8f-8561-e466398793c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:29.956721 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ba2b6-a26b-4a8f-8561-e466398793c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:29.956990 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ba2b6-a26b-4a8f-8561-e466398793c5 admin admin] PATH_INFO: `/v3/projects/fdcb1a9ee0554b15a189cdd5f7a14de0/users/ee3cd467a4104c9bbdaca3d791e1d8f4/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:29.972155 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ba2b6-a26b-4a8f-8561-e466398793c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdcb1a9ee0554b15a189cdd5f7a14de0, user_id=ee3cd467a4104c9bbdaca3d791e1d8f4, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:29.975085 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ba2b6-a26b-4a8f-8561-e466398793c5 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:29.987365 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 927/1874] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:29 2022] PUT /identity/v3/projects/fdcb1a9ee0554b15a189cdd5f7a14de0/users/ee3cd467a4104c9bbdaca3d791e1d8f4/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:29.991294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dfe685-3cce-4069-a635-4faabeca54f0 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:30.001329 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dfe685-3cce-4069-a635-4faabeca54f0 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:30.056228 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5734407-1d07-4a7c-b098-6c2f6254c434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:30.056826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5734407-1d07-4a7c-b098-6c2f6254c434 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:30.057105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5734407-1d07-4a7c-b098-6c2f6254c434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:30.057380 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5734407-1d07-4a7c-b098-6c2f6254c434 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:30.058038 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5734407-1d07-4a7c-b098-6c2f6254c434 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:30.059629 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5734407-1d07-4a7c-b098-6c2f6254c434 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:30.069041 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 948/1875] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:30.073599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ba2b6-a26b-4a8f-8561-e466398793c5 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:30.083693 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ba2b6-a26b-4a8f-8561-e466398793c5 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:30.088230 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29dff1-1095-4705-a223-4e1da4ad43cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:30.089490 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29dff1-1095-4705-a223-4e1da4ad43cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:30.089763 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29dff1-1095-4705-a223-4e1da4ad43cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:30.090113 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29dff1-1095-4705-a223-4e1da4ad43cc admin admin] PATH_INFO: `/v3/projects/fdcb1a9ee0554b15a189cdd5f7a14de0/users/ee3cd467a4104c9bbdaca3d791e1d8f4/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:30.092135 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd29dff1-1095-4705-a223-4e1da4ad43cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdcb1a9ee0554b15a189cdd5f7a14de0, user_id=ee3cd467a4104c9bbdaca3d791e1d8f4, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:30.094754 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd29dff1-1095-4705-a223-4e1da4ad43cc admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:30.105178 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 928/1876] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:30 2022] PUT /identity/v3/projects/fdcb1a9ee0554b15a189cdd5f7a14de0/users/ee3cd467a4104c9bbdaca3d791e1d8f4/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:30.115027 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f77f5-f374-4afa-a27b-07973f672746 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:30.115315 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f77f5-f374-4afa-a27b-07973f672746 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:30.115587 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f77f5-f374-4afa-a27b-07973f672746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:30.145996 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-a52f77f5-f374-4afa-a27b-07973f672746 None None] MFA Rules not processed for user `ee3cd467a4104c9bbdaca3d791e1d8f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:30.147284 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-a52f77f5-f374-4afa-a27b-07973f672746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:30.210284 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:30.236740 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 949/1877] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:30 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:30.244128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29dff1-1095-4705-a223-4e1da4ad43cc admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:30.246237 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-fd29dff1-1095-4705-a223-4e1da4ad43cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:30.317270 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29dff1-1095-4705-a223-4e1da4ad43cc admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:30.321475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5610ac-027c-40d8-ab20-a25be63aab35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:30.322230 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5610ac-027c-40d8-ab20-a25be63aab35 service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:30.322569 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5610ac-027c-40d8-ab20-a25be63aab35 service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:30.322899 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5610ac-027c-40d8-ab20-a25be63aab35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:30.333920 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5610ac-027c-40d8-ab20-a25be63aab35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:30.335489 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5610ac-027c-40d8-ab20-a25be63aab35 service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:30.350853 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 929/1878] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:30 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:33:30.471559 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f77f5-f374-4afa-a27b-07973f672746 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:30.484273 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f77f5-f374-4afa-a27b-07973f672746 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:30.488446 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c6dfd-4338-41ce-a9a4-4eebd712ad67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:30.489431 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c6dfd-4338-41ce-a9a4-4eebd712ad67 service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:30.489910 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c6dfd-4338-41ce-a9a4-4eebd712ad67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:30.490334 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c6dfd-4338-41ce-a9a4-4eebd712ad67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:30.564092 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26c6dfd-4338-41ce-a9a4-4eebd712ad67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:30.565524 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26c6dfd-4338-41ce-a9a4-4eebd712ad67 service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:30.581031 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 950/1879] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:30 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:33:40.342125 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=87,keystone:INSERT=5 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:40.570777 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=33 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:40.732648 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5610ac-027c-40d8-ab20-a25be63aab35 service cinder] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:40.819100 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5610ac-027c-40d8-ab20-a25be63aab35 service cinder] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:40.824646 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304350d-6043-4bf8-af6d-63cb2e11aac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:40.825363 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304350d-6043-4bf8-af6d-63cb2e11aac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:40.825715 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304350d-6043-4bf8-af6d-63cb2e11aac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:40.826677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304350d-6043-4bf8-af6d-63cb2e11aac4 admin admin] PATH_INFO: `/v3/users/6b7abedf87624d06acc2260bd6b9838f` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:40.827315 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f304350d-6043-4bf8-af6d-63cb2e11aac4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b7abedf87624d06acc2260bd6b9838f)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:40.829300 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f304350d-6043-4bf8-af6d-63cb2e11aac4 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:40.891307 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-f304350d-6043-4bf8-af6d-63cb2e11aac4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b7abedf87624d06acc2260bd6b9838f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:40.892458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-f304350d-6043-4bf8-af6d-63cb2e11aac4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b7abedf87624d06acc2260bd6b9838f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:40.893303 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-f304350d-6043-4bf8-af6d-63cb2e11aac4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b7abedf87624d06acc2260bd6b9838f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:40.901958 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-f304350d-6043-4bf8-af6d-63cb2e11aac4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b7abedf87624d06acc2260bd6b9838f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:40.915845 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-f304350d-6043-4bf8-af6d-63cb2e11aac4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b7abedf87624d06acc2260bd6b9838f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:40.931956 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 930/1880] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:33:40 2022] DELETE /identity/v3/users/6b7abedf87624d06acc2260bd6b9838f => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:40.938345 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0649e4-9d6b-4562-a64a-6fc73df00df1 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:40.938715 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0649e4-9d6b-4562-a64a-6fc73df00df1 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:40.939118 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0649e4-9d6b-4562-a64a-6fc73df00df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:40.977478 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-0f0649e4-9d6b-4562-a64a-6fc73df00df1 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:40.979175 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-0f0649e4-9d6b-4562-a64a-6fc73df00df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:41.051543 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 951/1881] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:41.062114 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304350d-6043-4bf8-af6d-63cb2e11aac4 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:41.063653 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-f304350d-6043-4bf8-af6d-63cb2e11aac4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:41.130470 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304350d-6043-4bf8-af6d-63cb2e11aac4 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:41.134237 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef48216-68e6-47c4-b48c-ac0f94f183a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:41.134853 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef48216-68e6-47c4-b48c-ac0f94f183a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:41.135134 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef48216-68e6-47c4-b48c-ac0f94f183a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:41.135415 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef48216-68e6-47c4-b48c-ac0f94f183a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:41.147641 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef48216-68e6-47c4-b48c-ac0f94f183a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:41.149512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef48216-68e6-47c4-b48c-ac0f94f183a8 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:41.160577 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 931/1882] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:33:41.450080 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0649e4-9d6b-4562-a64a-6fc73df00df1 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:41.451679 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-0f0649e4-9d6b-4562-a64a-6fc73df00df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:41.465027 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0649e4-9d6b-4562-a64a-6fc73df00df1 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:41.469987 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f821eb1c-b343-43e6-9f1c-62ca7f8a4449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:41.470650 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821eb1c-b343-43e6-9f1c-62ca7f8a4449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:41.470927 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821eb1c-b343-43e6-9f1c-62ca7f8a4449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:41.471197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821eb1c-b343-43e6-9f1c-62ca7f8a4449 admin admin] PATH_INFO: `/v3/projects/72e30732b0864ce0b56fabdc16568d65` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:41.472329 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f821eb1c-b343-43e6-9f1c-62ca7f8a4449 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72e30732b0864ce0b56fabdc16568d65)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:41.474540 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f821eb1c-b343-43e6-9f1c-62ca7f8a4449 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:41.533214 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f821eb1c-b343-43e6-9f1c-62ca7f8a4449 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72e30732b0864ce0b56fabdc16568d65', 'request_id': 'req-f821eb1c-b343-43e6-9f1c-62ca7f8a4449'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:41.539608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f821eb1c-b343-43e6-9f1c-62ca7f8a4449 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72e30732b0864ce0b56fabdc16568d65', 'request_id': 'req-f821eb1c-b343-43e6-9f1c-62ca7f8a4449'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:41.548422 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f821eb1c-b343-43e6-9f1c-62ca7f8a4449 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72e30732b0864ce0b56fabdc16568d65', 'request_id': 'req-f821eb1c-b343-43e6-9f1c-62ca7f8a4449'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:41.560713 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f821eb1c-b343-43e6-9f1c-62ca7f8a4449 admin admin] The token cache is being invalidate because project 72e30732b0864ce0b56fabdc16568d65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:33:41.560713 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f821eb1c-b343-43e6-9f1c-62ca7f8a4449 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:41.561343 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f821eb1c-b343-43e6-9f1c-62ca7f8a4449 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:41.563177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 952/1883] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:33:41 2022] DELETE /identity/v3/projects/72e30732b0864ce0b56fabdc16568d65 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:33:41.567123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef48216-68e6-47c4-b48c-ac0f94f183a8 service neutron] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:41.578381 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef48216-68e6-47c4-b48c-ac0f94f183a8 service neutron] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:41.599548 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:41.646829 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d0ebc4-f510-411e-8348-133e7c01a287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:41.647466 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0ebc4-f510-411e-8348-133e7c01a287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:41.647744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0ebc4-f510-411e-8348-133e7c01a287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:41.648071 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0ebc4-f510-411e-8348-133e7c01a287 admin admin] PATH_INFO: `/v3/users/ee3cd467a4104c9bbdaca3d791e1d8f4` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:41.649432 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d0ebc4-f510-411e-8348-133e7c01a287 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee3cd467a4104c9bbdaca3d791e1d8f4)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:41.655170 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d0ebc4-f510-411e-8348-133e7c01a287 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:41.717292 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-43d0ebc4-f510-411e-8348-133e7c01a287 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee3cd467a4104c9bbdaca3d791e1d8f4'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:41.717969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-43d0ebc4-f510-411e-8348-133e7c01a287 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee3cd467a4104c9bbdaca3d791e1d8f4'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:41.718688 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-43d0ebc4-f510-411e-8348-133e7c01a287 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee3cd467a4104c9bbdaca3d791e1d8f4'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:41.727849 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-43d0ebc4-f510-411e-8348-133e7c01a287 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee3cd467a4104c9bbdaca3d791e1d8f4'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:41.754141 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-43d0ebc4-f510-411e-8348-133e7c01a287 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee3cd467a4104c9bbdaca3d791e1d8f4'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:41.768965 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 932/1884] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:33:41 2022] DELETE /identity/v3/users/ee3cd467a4104c9bbdaca3d791e1d8f4 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:42.363874 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f821eb1c-b343-43e6-9f1c-62ca7f8a4449 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:42.364625 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-f821eb1c-b343-43e6-9f1c-62ca7f8a4449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:42.397600 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f821eb1c-b343-43e6-9f1c-62ca7f8a4449 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:42.487926 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d48723a-09e5-4996-9d2e-bfbba85e8352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:42.488854 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48723a-09e5-4996-9d2e-bfbba85e8352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:42.489198 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48723a-09e5-4996-9d2e-bfbba85e8352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:42.489602 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48723a-09e5-4996-9d2e-bfbba85e8352 admin admin] PATH_INFO: `/v3/projects/fdcb1a9ee0554b15a189cdd5f7a14de0` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:42.492212 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d48723a-09e5-4996-9d2e-bfbba85e8352 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdcb1a9ee0554b15a189cdd5f7a14de0)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:42.494795 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d48723a-09e5-4996-9d2e-bfbba85e8352 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:42.548740 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:42.613871 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-0d48723a-09e5-4996-9d2e-bfbba85e8352 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdcb1a9ee0554b15a189cdd5f7a14de0', 'request_id': 'req-0d48723a-09e5-4996-9d2e-bfbba85e8352'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:42.629528 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-0d48723a-09e5-4996-9d2e-bfbba85e8352 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdcb1a9ee0554b15a189cdd5f7a14de0', 'request_id': 'req-0d48723a-09e5-4996-9d2e-bfbba85e8352'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:42.652030 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-0d48723a-09e5-4996-9d2e-bfbba85e8352 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdcb1a9ee0554b15a189cdd5f7a14de0', 'request_id': 'req-0d48723a-09e5-4996-9d2e-bfbba85e8352'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:42.666297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-0d48723a-09e5-4996-9d2e-bfbba85e8352 admin admin] The token cache is being invalidate because project fdcb1a9ee0554b15a189cdd5f7a14de0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:33:42.666672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-0d48723a-09e5-4996-9d2e-bfbba85e8352 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:42.667797 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-0d48723a-09e5-4996-9d2e-bfbba85e8352 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:42.669897 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 953/1885] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:33:42 2022] DELETE /identity/v3/projects/fdcb1a9ee0554b15a189cdd5f7a14de0 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:42.674543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff7bb8-2152-4470-8074-938765779eda None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:42.675000 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff7bb8-2152-4470-8074-938765779eda None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:42.675083 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff7bb8-2152-4470-8074-938765779eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:42.704154 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-97ff7bb8-2152-4470-8074-938765779eda None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:42.705388 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-97ff7bb8-2152-4470-8074-938765779eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:42.780255 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 933/1886] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:42.787698 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57a3b2-5a39-4726-aa39-a73950a6cb8d None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:42.787991 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57a3b2-5a39-4726-aa39-a73950a6cb8d None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:42.788268 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57a3b2-5a39-4726-aa39-a73950a6cb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:42.797854 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff7bb8-2152-4470-8074-938765779eda None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:42.799257 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-97ff7bb8-2152-4470-8074-938765779eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:42.813100 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff7bb8-2152-4470-8074-938765779eda None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:42.813274 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-bd57a3b2-5a39-4726-aa39-a73950a6cb8d None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:42.814456 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-bd57a3b2-5a39-4726-aa39-a73950a6cb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:42.817985 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7e113-8996-41b9-b396-c4dd4e7f6c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:42.818643 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7e113-8996-41b9-b396-c4dd4e7f6c0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:42.819070 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7e113-8996-41b9-b396-c4dd4e7f6c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:42.819348 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7e113-8996-41b9-b396-c4dd4e7f6c0c admin admin] PATH_INFO: `/v3/users/39ead3dfe9cf4788b66309ecefec6ca8` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:42.820486 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba7e113-8996-41b9-b396-c4dd4e7f6c0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=39ead3dfe9cf4788b66309ecefec6ca8)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:42.822436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba7e113-8996-41b9-b396-c4dd4e7f6c0c admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:42.822568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 954/1887] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:42.826641 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd57a3b2-5a39-4726-aa39-a73950a6cb8d None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:42.835706 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd57a3b2-5a39-4726-aa39-a73950a6cb8d None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:42.840282 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b923c-84b6-461e-885b-515b43af8dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:42.840868 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b923c-84b6-461e-885b-515b43af8dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:42.841146 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b923c-84b6-461e-885b-515b43af8dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:42.841422 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b923c-84b6-461e-885b-515b43af8dd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:42.842090 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594b923c-84b6-461e-885b-515b43af8dd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:33:42.842393 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594b923c-84b6-461e-885b-515b43af8dd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:42.843973 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594b923c-84b6-461e-885b-515b43af8dd4 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:42.857824 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 955/1888] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:33:42.862383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b923c-84b6-461e-885b-515b43af8dd4 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:42.862914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:42.874250 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-1ba7e113-8996-41b9-b396-c4dd4e7f6c0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39ead3dfe9cf4788b66309ecefec6ca8'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:42.874714 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b923c-84b6-461e-885b-515b43af8dd4 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:42.874917 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-1ba7e113-8996-41b9-b396-c4dd4e7f6c0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39ead3dfe9cf4788b66309ecefec6ca8'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:42.875570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-1ba7e113-8996-41b9-b396-c4dd4e7f6c0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39ead3dfe9cf4788b66309ecefec6ca8'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:42.879640 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f7880-913b-4e3f-b89f-52ed4cf21282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:42.880229 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f7880-913b-4e3f-b89f-52ed4cf21282 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:42.880504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f7880-913b-4e3f-b89f-52ed4cf21282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:42.880767 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f7880-913b-4e3f-b89f-52ed4cf21282 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:42.881398 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f7880-913b-4e3f-b89f-52ed4cf21282 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:42.882841 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-1ba7e113-8996-41b9-b396-c4dd4e7f6c0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39ead3dfe9cf4788b66309ecefec6ca8'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:42.883481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f7880-913b-4e3f-b89f-52ed4cf21282 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:42.895055 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-1ba7e113-8996-41b9-b396-c4dd4e7f6c0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39ead3dfe9cf4788b66309ecefec6ca8'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:42.909080 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 934/1889] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:33:42 2022] DELETE /identity/v3/users/39ead3dfe9cf4788b66309ecefec6ca8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:33:42.913563 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c999894-291a-4ced-b55f-df87e53de09b None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:42.913854 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c999894-291a-4ced-b55f-df87e53de09b None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:42.914162 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c999894-291a-4ced-b55f-df87e53de09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:42.915094 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 956/1890] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:33:42 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:33:42.919378 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f7880-913b-4e3f-b89f-52ed4cf21282 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:42.920994 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-6d1f7880-913b-4e3f-b89f-52ed4cf21282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:42.935426 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-0c999894-291a-4ced-b55f-df87e53de09b None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:42.936571 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-0c999894-291a-4ced-b55f-df87e53de09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:42.999073 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f7880-913b-4e3f-b89f-52ed4cf21282 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:43.004942 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a86871-d08c-43e1-8acb-76eb2ed8950d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:43.005712 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a86871-d08c-43e1-8acb-76eb2ed8950d admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:43.006097 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a86871-d08c-43e1-8acb-76eb2ed8950d admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:43.006529 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a86871-d08c-43e1-8acb-76eb2ed8950d admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:43.007308 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a86871-d08c-43e1-8acb-76eb2ed8950d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:43.008205 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 935/1891] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:33:43.012112 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a86871-d08c-43e1-8acb-76eb2ed8950d admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:43.018962 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c999894-291a-4ced-b55f-df87e53de09b None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:43.020329 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-0c999894-291a-4ced-b55f-df87e53de09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:43.034156 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 957/1892] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:33:42 2022] POST /identity/v3/users => generated 283 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:33:43.038318 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a86871-d08c-43e1-8acb-76eb2ed8950d admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:43.048701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a86871-d08c-43e1-8acb-76eb2ed8950d admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:43.053256 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc0cfb-144b-4954-870a-a143e8bb32d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:43.053873 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc0cfb-144b-4954-870a-a143e8bb32d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:43.054185 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc0cfb-144b-4954-870a-a143e8bb32d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:43.054461 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc0cfb-144b-4954-870a-a143e8bb32d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:43.055097 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc0cfb-144b-4954-870a-a143e8bb32d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:43.056722 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc0cfb-144b-4954-870a-a143e8bb32d8 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:43.069067 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 958/1893] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:33:43.072392 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc0cfb-144b-4954-870a-a143e8bb32d8 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:43.086649 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc0cfb-144b-4954-870a-a143e8bb32d8 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:43.087454 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c999894-291a-4ced-b55f-df87e53de09b None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:43.092717 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8284c2-cdf8-4043-81e6-2866c8075577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:43.093273 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ea907-abf2-490b-bcbf-ea8416f35316 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:43.093518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8284c2-cdf8-4043-81e6-2866c8075577 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:43.093795 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8284c2-cdf8-4043-81e6-2866c8075577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:43.094151 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ea907-abf2-490b-bcbf-ea8416f35316 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:43.094427 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ea907-abf2-490b-bcbf-ea8416f35316 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:43.094694 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ea907-abf2-490b-bcbf-ea8416f35316 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:43.094919 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8284c2-cdf8-4043-81e6-2866c8075577 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:43.095571 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8284c2-cdf8-4043-81e6-2866c8075577 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:43.097183 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8284c2-cdf8-4043-81e6-2866c8075577 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:43.105349 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ea907-abf2-490b-bcbf-ea8416f35316 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:43.107397 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ea907-abf2-490b-bcbf-ea8416f35316 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:43.114469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 959/1894] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:43.115557 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8284c2-cdf8-4043-81e6-2866c8075577 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:43.117695 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 936/1895] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:33:43.127710 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8284c2-cdf8-4043-81e6-2866c8075577 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:43.132377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59930f3b-77e2-471b-a101-77e4ca6a4792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:43.133036 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59930f3b-77e2-471b-a101-77e4ca6a4792 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:43.133314 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59930f3b-77e2-471b-a101-77e4ca6a4792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:43.133591 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59930f3b-77e2-471b-a101-77e4ca6a4792 admin admin] PATH_INFO: `/v3/projects/9b241373837649d19fc60b7220ae228e/users/d14e900f3aac4c889a401571450118a3/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:43.149137 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59930f3b-77e2-471b-a101-77e4ca6a4792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b241373837649d19fc60b7220ae228e, user_id=d14e900f3aac4c889a401571450118a3, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:43.152387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59930f3b-77e2-471b-a101-77e4ca6a4792 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:43.166317 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 960/1896] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:43 2022] PUT /identity/v3/projects/9b241373837649d19fc60b7220ae228e/users/d14e900f3aac4c889a401571450118a3/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:43.178680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf83c8-dce3-49a9-990b-11218a2ad9f7 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:43.178961 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf83c8-dce3-49a9-990b-11218a2ad9f7 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:43.179324 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf83c8-dce3-49a9-990b-11218a2ad9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:43.210922 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-b6bf83c8-dce3-49a9-990b-11218a2ad9f7 None None] MFA Rules not processed for user `d14e900f3aac4c889a401571450118a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:43.212282 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-b6bf83c8-dce3-49a9-990b-11218a2ad9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:43.281456 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:43.319448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 937/1897] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:43.325908 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59930f3b-77e2-471b-a101-77e4ca6a4792 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:43.361202 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:DELETE=6,keystone:SELECT=88,keystone:INSERT=6 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:43.435239 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59930f3b-77e2-471b-a101-77e4ca6a4792 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:43.445924 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be98ffc-dfaa-40d9-b572-eb607c572133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:43.448965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be98ffc-dfaa-40d9-b572-eb607c572133 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:43.449345 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be98ffc-dfaa-40d9-b572-eb607c572133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:43.450014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be98ffc-dfaa-40d9-b572-eb607c572133 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:43.451028 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be98ffc-dfaa-40d9-b572-eb607c572133 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:43.453526 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be98ffc-dfaa-40d9-b572-eb607c572133 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:43.505568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 961/1898] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:33:43 2022] POST /identity/v3/projects => generated 367 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:43.507348 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf83c8-dce3-49a9-990b-11218a2ad9f7 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:43.523182 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf83c8-dce3-49a9-990b-11218a2ad9f7 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:43.584362 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be98ffc-dfaa-40d9-b572-eb607c572133 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:43.586347 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-0be98ffc-dfaa-40d9-b572-eb607c572133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:43.612213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce05f94-11fd-4fa4-8813-084196dbb0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:43.613075 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce05f94-11fd-4fa4-8813-084196dbb0ff admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:43.613566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce05f94-11fd-4fa4-8813-084196dbb0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:43.613929 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce05f94-11fd-4fa4-8813-084196dbb0ff admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:43.614688 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce05f94-11fd-4fa4-8813-084196dbb0ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:43.616779 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce05f94-11fd-4fa4-8813-084196dbb0ff admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:43.647821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 938/1899] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:33:43 2022] POST /identity/v3/users => generated 290 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:43.654396 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce05f94-11fd-4fa4-8813-084196dbb0ff admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:43.669843 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce05f94-11fd-4fa4-8813-084196dbb0ff admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:43.675604 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea066172-c51f-4c80-9974-f80af734d776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:43.676567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea066172-c51f-4c80-9974-f80af734d776 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:43.677054 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea066172-c51f-4c80-9974-f80af734d776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:43.677343 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea066172-c51f-4c80-9974-f80af734d776 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:43.678118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea066172-c51f-4c80-9974-f80af734d776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:43.680277 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea066172-c51f-4c80-9974-f80af734d776 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:43.686014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be98ffc-dfaa-40d9-b572-eb607c572133 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:43.691973 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8675f52-dce6-4f68-b277-4280c930b794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:43.692619 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8675f52-dce6-4f68-b277-4280c930b794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:43.692902 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8675f52-dce6-4f68-b277-4280c930b794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:43.693371 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 939/1900] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:43.694519 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8675f52-dce6-4f68-b277-4280c930b794 admin admin] PATH_INFO: `/v3/projects/08019dd7d2454aceb518d7db58ee3278` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:43.695951 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8675f52-dce6-4f68-b277-4280c930b794 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08019dd7d2454aceb518d7db58ee3278)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:43.700247 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8675f52-dce6-4f68-b277-4280c930b794 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:43.703037 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea066172-c51f-4c80-9974-f80af734d776 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:43.723093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea066172-c51f-4c80-9974-f80af734d776 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:43.740662 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48c433-7a9a-4da4-a943-66f4d397736f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:43.741680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48c433-7a9a-4da4-a943-66f4d397736f admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:43.742070 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48c433-7a9a-4da4-a943-66f4d397736f admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:43.742520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48c433-7a9a-4da4-a943-66f4d397736f admin admin] PATH_INFO: `/v3/projects/5b434968ce3f4982a0c05ba45593b3b9/users/8f40290cab2147fc83b46bc70401a669/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:43.775822 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e48c433-7a9a-4da4-a943-66f4d397736f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b434968ce3f4982a0c05ba45593b3b9, user_id=8f40290cab2147fc83b46bc70401a669, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:43.780915 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e48c433-7a9a-4da4-a943-66f4d397736f admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:43.787357 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d8675f52-dce6-4f68-b277-4280c930b794 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08019dd7d2454aceb518d7db58ee3278', 'request_id': 'req-d8675f52-dce6-4f68-b277-4280c930b794'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:43.802949 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d8675f52-dce6-4f68-b277-4280c930b794 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08019dd7d2454aceb518d7db58ee3278', 'request_id': 'req-d8675f52-dce6-4f68-b277-4280c930b794'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:43.817891 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 940/1901] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:43 2022] PUT /identity/v3/projects/5b434968ce3f4982a0c05ba45593b3b9/users/8f40290cab2147fc83b46bc70401a669/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:43.821916 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48c433-7a9a-4da4-a943-66f4d397736f admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:43.825125 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d8675f52-dce6-4f68-b277-4280c930b794 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08019dd7d2454aceb518d7db58ee3278', 'request_id': 'req-d8675f52-dce6-4f68-b277-4280c930b794'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:43.844021 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d8675f52-dce6-4f68-b277-4280c930b794 admin admin] The token cache is being invalidate because project 08019dd7d2454aceb518d7db58ee3278 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:33:43.844410 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d8675f52-dce6-4f68-b277-4280c930b794 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:43.845303 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d8675f52-dce6-4f68-b277-4280c930b794 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:43.847523 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 962/1902] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:33:43 2022] DELETE /identity/v3/projects/08019dd7d2454aceb518d7db58ee3278 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:43.851711 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8675f52-dce6-4f68-b277-4280c930b794 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:43.854190 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-d8675f52-dce6-4f68-b277-4280c930b794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:43.895581 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:43.904052 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48c433-7a9a-4da4-a943-66f4d397736f admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:43.910096 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857530da-64da-4414-a317-ce53d1720bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:43.911066 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857530da-64da-4414-a317-ce53d1720bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:43.911335 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857530da-64da-4414-a317-ce53d1720bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:43.911805 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857530da-64da-4414-a317-ce53d1720bc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:43.912486 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857530da-64da-4414-a317-ce53d1720bc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:43.914053 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857530da-64da-4414-a317-ce53d1720bc3 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:43.927072 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 941/1903] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:43.930529 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857530da-64da-4414-a317-ce53d1720bc3 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:43.932225 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-857530da-64da-4414-a317-ce53d1720bc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:43.939598 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8675f52-dce6-4f68-b277-4280c930b794 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:43.945328 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5976ae6-e79a-4326-93e1-3ee42ddb79c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:43.946454 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5976ae6-e79a-4326-93e1-3ee42ddb79c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:43.946908 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857530da-64da-4414-a317-ce53d1720bc3 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:43.947075 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5976ae6-e79a-4326-93e1-3ee42ddb79c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:43.947566 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5976ae6-e79a-4326-93e1-3ee42ddb79c7 admin admin] PATH_INFO: `/v3/users/236eb289b08544a0ae17ebd5c353b4a7` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:43.948787 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5976ae6-e79a-4326-93e1-3ee42ddb79c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=236eb289b08544a0ae17ebd5c353b4a7)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:43.950882 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5976ae6-e79a-4326-93e1-3ee42ddb79c7 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:43.951802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc76c8c-8d76-4973-96b9-102a3a1e1235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:43.952449 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc76c8c-8d76-4973-96b9-102a3a1e1235 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:43.952823 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc76c8c-8d76-4973-96b9-102a3a1e1235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:43.953104 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc76c8c-8d76-4973-96b9-102a3a1e1235 admin admin] PATH_INFO: `/v3/projects/5b434968ce3f4982a0c05ba45593b3b9/users/8f40290cab2147fc83b46bc70401a669/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:43.955060 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc76c8c-8d76-4973-96b9-102a3a1e1235 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b434968ce3f4982a0c05ba45593b3b9, user_id=8f40290cab2147fc83b46bc70401a669, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:43.957766 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc76c8c-8d76-4973-96b9-102a3a1e1235 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:43.970485 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 942/1904] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:43 2022] PUT /identity/v3/projects/5b434968ce3f4982a0c05ba45593b3b9/users/8f40290cab2147fc83b46bc70401a669/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:43.981555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc0b40-7960-4eda-87a8-ba7ec4fa37c6 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:43.981923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc0b40-7960-4eda-87a8-ba7ec4fa37c6 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:43.982205 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc0b40-7960-4eda-87a8-ba7ec4fa37c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:44.012131 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-e5976ae6-e79a-4326-93e1-3ee42ddb79c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '236eb289b08544a0ae17ebd5c353b4a7'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:44.012712 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-88bc0b40-7960-4eda-87a8-ba7ec4fa37c6 None None] MFA Rules not processed for user `8f40290cab2147fc83b46bc70401a669`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:44.012911 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-e5976ae6-e79a-4326-93e1-3ee42ddb79c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '236eb289b08544a0ae17ebd5c353b4a7'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:44.013985 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-88bc0b40-7960-4eda-87a8-ba7ec4fa37c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:44.020352 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-e5976ae6-e79a-4326-93e1-3ee42ddb79c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '236eb289b08544a0ae17ebd5c353b4a7'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:44.026145 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:44.033236 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-e5976ae6-e79a-4326-93e1-3ee42ddb79c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '236eb289b08544a0ae17ebd5c353b4a7'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:44.033927 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-e5976ae6-e79a-4326-93e1-3ee42ddb79c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '236eb289b08544a0ae17ebd5c353b4a7'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:44.047785 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 963/1905] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:33:43 2022] DELETE /identity/v3/users/236eb289b08544a0ae17ebd5c353b4a7 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:33:44.092318 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 943/1906] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:43 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:33:44.106066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5976ae6-e79a-4326-93e1-3ee42ddb79c7 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:44.107583 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-e5976ae6-e79a-4326-93e1-3ee42ddb79c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:44.169509 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5976ae6-e79a-4326-93e1-3ee42ddb79c7 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:44.173235 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd61176-05a0-4f6b-be6f-e266867a528f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:44.173916 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd61176-05a0-4f6b-be6f-e266867a528f service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:44.174288 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd61176-05a0-4f6b-be6f-e266867a528f service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:44.174626 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd61176-05a0-4f6b-be6f-e266867a528f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:44.249975 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd61176-05a0-4f6b-be6f-e266867a528f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:44.251668 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd61176-05a0-4f6b-be6f-e266867a528f service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:44.256968 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:44.266244 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 964/1907] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:44 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:33:44.365806 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc0b40-7960-4eda-87a8-ba7ec4fa37c6 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:44.367862 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-88bc0b40-7960-4eda-87a8-ba7ec4fa37c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:44.397585 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd61176-05a0-4f6b-be6f-e266867a528f service cinder] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:44.400487 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-6bd61176-05a0-4f6b-be6f-e266867a528f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:44.479326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc0b40-7960-4eda-87a8-ba7ec4fa37c6 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:44.486196 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f32fa83-cd8e-4b6d-8420-581dbf55161f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:44.488334 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32fa83-cd8e-4b6d-8420-581dbf55161f service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:44.489271 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32fa83-cd8e-4b6d-8420-581dbf55161f service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:44.489805 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32fa83-cd8e-4b6d-8420-581dbf55161f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:44.492055 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-3f32fa83-cd8e-4b6d-8420-581dbf55161f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:44.560141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd61176-05a0-4f6b-be6f-e266867a528f service cinder] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:44.567200 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e674b0-0655-49a1-b6d6-4f5b3794d07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:44.567200 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e674b0-0655-49a1-b6d6-4f5b3794d07d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:44.567200 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e674b0-0655-49a1-b6d6-4f5b3794d07d admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:44.567200 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e674b0-0655-49a1-b6d6-4f5b3794d07d admin admin] PATH_INFO: `/v3/projects/934748f0880f4d4e9c6eea10fdb2f4a8` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:44.569098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e674b0-0655-49a1-b6d6-4f5b3794d07d admin admin] RBAC: Authorizing `identity:delete_project(project_id=934748f0880f4d4e9c6eea10fdb2f4a8)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:44.571214 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e674b0-0655-49a1-b6d6-4f5b3794d07d admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:44.585038 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f32fa83-cd8e-4b6d-8420-581dbf55161f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:44.586641 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f32fa83-cd8e-4b6d-8420-581dbf55161f service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:44.598620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 944/1908] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:33:44.659200 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c1e674b0-0655-49a1-b6d6-4f5b3794d07d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '934748f0880f4d4e9c6eea10fdb2f4a8', 'request_id': 'req-c1e674b0-0655-49a1-b6d6-4f5b3794d07d'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:44.690268 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c1e674b0-0655-49a1-b6d6-4f5b3794d07d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '934748f0880f4d4e9c6eea10fdb2f4a8', 'request_id': 'req-c1e674b0-0655-49a1-b6d6-4f5b3794d07d'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:44.695258 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c1e674b0-0655-49a1-b6d6-4f5b3794d07d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '934748f0880f4d4e9c6eea10fdb2f4a8', 'request_id': 'req-c1e674b0-0655-49a1-b6d6-4f5b3794d07d'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:44.709708 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c1e674b0-0655-49a1-b6d6-4f5b3794d07d admin admin] The token cache is being invalidate because project 934748f0880f4d4e9c6eea10fdb2f4a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:33:44.710123 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c1e674b0-0655-49a1-b6d6-4f5b3794d07d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:44.711736 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c1e674b0-0655-49a1-b6d6-4f5b3794d07d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:44.713756 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 965/1909] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:33:44 2022] DELETE /identity/v3/projects/934748f0880f4d4e9c6eea10fdb2f4a8 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:44.718967 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b704ae5-b2b4-4f66-ad34-297ce3462a5e None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:44.719600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b704ae5-b2b4-4f66-ad34-297ce3462a5e None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:44.719879 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b704ae5-b2b4-4f66-ad34-297ce3462a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:44.741460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-3b704ae5-b2b4-4f66-ad34-297ce3462a5e None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:44.742731 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-3b704ae5-b2b4-4f66-ad34-297ce3462a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:44.836865 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 945/1910] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:44.839825 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe86301-e96a-440e-87cd-c598c937f689 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:44.840206 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe86301-e96a-440e-87cd-c598c937f689 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:44.854112 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe86301-e96a-440e-87cd-c598c937f689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:44.864940 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-7fe86301-e96a-440e-87cd-c598c937f689 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:44.866342 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-7fe86301-e96a-440e-87cd-c598c937f689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:44.880258 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 966/1911] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:33:44.881731 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b704ae5-b2b4-4f66-ad34-297ce3462a5e None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:44.889852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:44.894373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b704ae5-b2b4-4f66-ad34-297ce3462a5e None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:44.900424 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f40de7-982b-420a-80d4-ff023309a0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:44.900740 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f40de7-982b-420a-80d4-ff023309a0d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:44.901888 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f40de7-982b-420a-80d4-ff023309a0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:44.902209 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f40de7-982b-420a-80d4-ff023309a0d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:44.902877 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f40de7-982b-420a-80d4-ff023309a0d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:33:44.903268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f40de7-982b-420a-80d4-ff023309a0d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:44.905732 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f40de7-982b-420a-80d4-ff023309a0d4 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:44.926058 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe86301-e96a-440e-87cd-c598c937f689 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:44.930704 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 946/1912] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:33:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:33:44.936671 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe86301-e96a-440e-87cd-c598c937f689 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:44.952666 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f971dc-5b9b-49fd-bc88-2c2bec2c0a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:44.953355 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f971dc-5b9b-49fd-bc88-2c2bec2c0a51 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:44.953649 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f971dc-5b9b-49fd-bc88-2c2bec2c0a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:44.953926 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f971dc-5b9b-49fd-bc88-2c2bec2c0a51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:44.955425 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f971dc-5b9b-49fd-bc88-2c2bec2c0a51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:44.959968 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f971dc-5b9b-49fd-bc88-2c2bec2c0a51 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:45.029111 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f40de7-982b-420a-80d4-ff023309a0d4 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:45.034566 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 967/1913] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:33:44 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:45.044257 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f971dc-5b9b-49fd-bc88-2c2bec2c0a51 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:45.047432 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f40de7-982b-420a-80d4-ff023309a0d4 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:45.049247 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-c6f971dc-5b9b-49fd-bc88-2c2bec2c0a51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:45.185509 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae167f-dc74-4901-a8e2-2f0db0f6242e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:45.186604 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae167f-dc74-4901-a8e2-2f0db0f6242e admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:45.187520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae167f-dc74-4901-a8e2-2f0db0f6242e admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:45.187520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae167f-dc74-4901-a8e2-2f0db0f6242e admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:45.189875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae167f-dc74-4901-a8e2-2f0db0f6242e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:45.192743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae167f-dc74-4901-a8e2-2f0db0f6242e admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:45.196051 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f971dc-5b9b-49fd-bc88-2c2bec2c0a51 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:45.201276 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d060a-e99b-4017-a3a2-fa76c07b11c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:45.201934 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d060a-e99b-4017-a3a2-fa76c07b11c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:45.202272 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d060a-e99b-4017-a3a2-fa76c07b11c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:45.202549 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d060a-e99b-4017-a3a2-fa76c07b11c0 admin admin] PATH_INFO: `/v3/users/74226ce13ced49129d822ba92b1a9165` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:45.206130 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89d060a-e99b-4017-a3a2-fa76c07b11c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74226ce13ced49129d822ba92b1a9165)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:45.208652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89d060a-e99b-4017-a3a2-fa76c07b11c0 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:45.212955 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:45.251774 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 947/1914] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:33:45 2022] POST /identity/v3/users => generated 293 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:33:45.258472 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae167f-dc74-4901-a8e2-2f0db0f6242e admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:45.285581 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae167f-dc74-4901-a8e2-2f0db0f6242e admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:45.299648 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af8e21-a660-4459-bb5d-b589867fb087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:45.301228 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-e89d060a-e99b-4017-a3a2-fa76c07b11c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74226ce13ced49129d822ba92b1a9165'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:45.301985 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-e89d060a-e99b-4017-a3a2-fa76c07b11c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74226ce13ced49129d822ba92b1a9165'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:45.302811 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af8e21-a660-4459-bb5d-b589867fb087 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:45.303672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af8e21-a660-4459-bb5d-b589867fb087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:45.304031 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af8e21-a660-4459-bb5d-b589867fb087 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:45.306122 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af8e21-a660-4459-bb5d-b589867fb087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:45.311531 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-e89d060a-e99b-4017-a3a2-fa76c07b11c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74226ce13ced49129d822ba92b1a9165'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:45.311992 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af8e21-a660-4459-bb5d-b589867fb087 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:45.329007 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-e89d060a-e99b-4017-a3a2-fa76c07b11c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74226ce13ced49129d822ba92b1a9165'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:45.330105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-e89d060a-e99b-4017-a3a2-fa76c07b11c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74226ce13ced49129d822ba92b1a9165'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:45.343927 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 948/1915] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:45 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:45.345932 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af8e21-a660-4459-bb5d-b589867fb087 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:45.350897 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 968/1916] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:33:45 2022] DELETE /identity/v3/users/74226ce13ced49129d822ba92b1a9165 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:45.352114 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-c5af8e21-a660-4459-bb5d-b589867fb087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:45.357345 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb984c85-7626-4e92-a28f-5713799b36d0 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:45.357931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb984c85-7626-4e92-a28f-5713799b36d0 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:45.358294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb984c85-7626-4e92-a28f-5713799b36d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:45.369569 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af8e21-a660-4459-bb5d-b589867fb087 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:45.408998 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-bb984c85-7626-4e92-a28f-5713799b36d0 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:45.410301 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-bb984c85-7626-4e92-a28f-5713799b36d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:45.473988 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59212912-6c98-4a4f-bb70-b7aa43c3b5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:45.475380 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59212912-6c98-4a4f-bb70-b7aa43c3b5cd admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:45.475795 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59212912-6c98-4a4f-bb70-b7aa43c3b5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:45.476093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59212912-6c98-4a4f-bb70-b7aa43c3b5cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:45.476652 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59212912-6c98-4a4f-bb70-b7aa43c3b5cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:45.478357 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59212912-6c98-4a4f-bb70-b7aa43c3b5cd admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:45.489583 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 949/1917] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:45 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:45.495525 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59212912-6c98-4a4f-bb70-b7aa43c3b5cd admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:45.507643 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59212912-6c98-4a4f-bb70-b7aa43c3b5cd admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:45.512419 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 969/1918] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:45.518244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c3c28-309a-4d8a-8c50-a511d2154947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:45.518812 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c3c28-309a-4d8a-8c50-a511d2154947 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:45.519105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c3c28-309a-4d8a-8c50-a511d2154947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:45.519439 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c3c28-309a-4d8a-8c50-a511d2154947 admin admin] PATH_INFO: `/v3/projects/b6fe68fa46be4280aa7633156c101706/users/82e1a8c14c3b40f08c889fad3d0d6076/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:45.524115 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb984c85-7626-4e92-a28f-5713799b36d0 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:45.526973 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-bb984c85-7626-4e92-a28f-5713799b36d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:45.539415 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c3c28-309a-4d8a-8c50-a511d2154947 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6fe68fa46be4280aa7633156c101706, user_id=82e1a8c14c3b40f08c889fad3d0d6076, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:45.543427 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c3c28-309a-4d8a-8c50-a511d2154947 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:45.562332 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 950/1919] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:45 2022] PUT /identity/v3/projects/b6fe68fa46be4280aa7633156c101706/users/82e1a8c14c3b40f08c889fad3d0d6076/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:33:45.576672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacc384-bdcd-462e-bcfb-ee72c858450a None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:45.577181 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacc384-bdcd-462e-bcfb-ee72c858450a None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:45.577821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacc384-bdcd-462e-bcfb-ee72c858450a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:45.591530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb984c85-7626-4e92-a28f-5713799b36d0 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:45.616144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-deacc384-bdcd-462e-bcfb-ee72c858450a None None] MFA Rules not processed for user `82e1a8c14c3b40f08c889fad3d0d6076`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:45.617374 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-deacc384-bdcd-462e-bcfb-ee72c858450a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:45.646097 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:45.659038 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:45.680224 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d5991-7e10-40e2-9651-97dda35aad18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:45.680774 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d5991-7e10-40e2-9651-97dda35aad18 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:45.681096 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d5991-7e10-40e2-9651-97dda35aad18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:45.681480 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d5991-7e10-40e2-9651-97dda35aad18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:45.709841 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 951/1920] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:33:45.714934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deacc384-bdcd-462e-bcfb-ee72c858450a None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:45.786897 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d5991-7e10-40e2-9651-97dda35aad18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:45.788370 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d5991-7e10-40e2-9651-97dda35aad18 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:45.814306 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deacc384-bdcd-462e-bcfb-ee72c858450a None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:45.817932 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 970/1921] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:33:45.819198 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb35e38-978d-4e26-b0c3-8fca6f4d869b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:45.819940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb35e38-978d-4e26-b0c3-8fca6f4d869b admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:45.820223 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb35e38-978d-4e26-b0c3-8fca6f4d869b admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:45.820497 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb35e38-978d-4e26-b0c3-8fca6f4d869b admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:45.821143 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb35e38-978d-4e26-b0c3-8fca6f4d869b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:45.823722 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb35e38-978d-4e26-b0c3-8fca6f4d869b admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:45.863345 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 952/1922] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:33:45 2022] POST /identity/v3/projects => generated 387 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:45.864488 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d5991-7e10-40e2-9651-97dda35aad18 service neutron] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:45.875281 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d5991-7e10-40e2-9651-97dda35aad18 service neutron] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:45.963674 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66507f8-e169-4bc8-8422-034ab7c8c7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:45.966062 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66507f8-e169-4bc8-8422-034ab7c8c7c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:45.966062 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66507f8-e169-4bc8-8422-034ab7c8c7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:45.966062 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66507f8-e169-4bc8-8422-034ab7c8c7c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:45.966505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66507f8-e169-4bc8-8422-034ab7c8c7c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:45.968508 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66507f8-e169-4bc8-8422-034ab7c8c7c4 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:45.995985 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 971/1923] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:33:45 2022] POST /identity/v3/users => generated 300 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:33:46.000524 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb35e38-978d-4e26-b0c3-8fca6f4d869b admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:46.012507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb35e38-978d-4e26-b0c3-8fca6f4d869b admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:46.017486 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d759eb93-db37-40cd-951a-f4bea1538409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:46.018200 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759eb93-db37-40cd-951a-f4bea1538409 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:46.018577 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759eb93-db37-40cd-951a-f4bea1538409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:46.018946 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759eb93-db37-40cd-951a-f4bea1538409 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:46.019691 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d759eb93-db37-40cd-951a-f4bea1538409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:46.021388 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d759eb93-db37-40cd-951a-f4bea1538409 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:46.033076 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 953/1924] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:33:46.037891 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66507f8-e169-4bc8-8422-034ab7c8c7c4 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:46.067916 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66507f8-e169-4bc8-8422-034ab7c8c7c4 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:46.076151 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27990daf-c8eb-4a59-b53c-571f0d694e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:46.076888 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27990daf-c8eb-4a59-b53c-571f0d694e59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:46.077221 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27990daf-c8eb-4a59-b53c-571f0d694e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:46.077503 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27990daf-c8eb-4a59-b53c-571f0d694e59 admin admin] PATH_INFO: `/v3/projects/e623ce4832a44325977a7d76f0d40c28/users/c258c18f7d2c41d8926f05a11a544902/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:46.095940 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27990daf-c8eb-4a59-b53c-571f0d694e59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e623ce4832a44325977a7d76f0d40c28, user_id=c258c18f7d2c41d8926f05a11a544902, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:46.099997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27990daf-c8eb-4a59-b53c-571f0d694e59 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:46.134555 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 972/1925] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:46 2022] PUT /identity/v3/projects/e623ce4832a44325977a7d76f0d40c28/users/c258c18f7d2c41d8926f05a11a544902/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:46.140084 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d759eb93-db37-40cd-951a-f4bea1538409 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:46.160064 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d759eb93-db37-40cd-951a-f4bea1538409 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:46.312185 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785ba45-6cea-4d9c-a3a6-327ca1d7cf5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:46.313025 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785ba45-6cea-4d9c-a3a6-327ca1d7cf5e admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:46.313401 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785ba45-6cea-4d9c-a3a6-327ca1d7cf5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:46.313774 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785ba45-6cea-4d9c-a3a6-327ca1d7cf5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:46.314687 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785ba45-6cea-4d9c-a3a6-327ca1d7cf5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:46.316464 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785ba45-6cea-4d9c-a3a6-327ca1d7cf5e admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:46.333614 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 954/1926] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:46.338643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27990daf-c8eb-4a59-b53c-571f0d694e59 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:46.368646 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27990daf-c8eb-4a59-b53c-571f0d694e59 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:46.376615 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a1637-5c79-48f4-9c91-98061f4d0793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:46.377549 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a1637-5c79-48f4-9c91-98061f4d0793 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:46.378117 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a1637-5c79-48f4-9c91-98061f4d0793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:46.378613 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a1637-5c79-48f4-9c91-98061f4d0793 admin admin] PATH_INFO: `/v3/projects/e623ce4832a44325977a7d76f0d40c28/users/c258c18f7d2c41d8926f05a11a544902/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:46.382691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a1637-5c79-48f4-9c91-98061f4d0793 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e623ce4832a44325977a7d76f0d40c28, user_id=c258c18f7d2c41d8926f05a11a544902, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:46.386920 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a1637-5c79-48f4-9c91-98061f4d0793 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:46.401410 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 973/1927] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:46 2022] PUT /identity/v3/projects/e623ce4832a44325977a7d76f0d40c28/users/c258c18f7d2c41d8926f05a11a544902/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:33:46.413589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358ddd9-ec8f-4a7d-b018-6b36054420ee None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:46.413999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358ddd9-ec8f-4a7d-b018-6b36054420ee None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:46.414323 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358ddd9-ec8f-4a7d-b018-6b36054420ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:46.500212 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-0358ddd9-ec8f-4a7d-b018-6b36054420ee None None] MFA Rules not processed for user `c258c18f7d2c41d8926f05a11a544902`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:46.508857 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a1637-5c79-48f4-9c91-98061f4d0793 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:46.509100 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-0358ddd9-ec8f-4a7d-b018-6b36054420ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:46.512027 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-a25a1637-5c79-48f4-9c91-98061f4d0793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:46.563597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:46.605314 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:46.622898 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 955/1928] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:46 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:46.634922 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0358ddd9-ec8f-4a7d-b018-6b36054420ee None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:46.637451 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-0358ddd9-ec8f-4a7d-b018-6b36054420ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:46.641414 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a1637-5c79-48f4-9c91-98061f4d0793 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:46.652192 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c29cf-c2ad-4f0a-8648-83de379e1570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:46.653524 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c29cf-c2ad-4f0a-8648-83de379e1570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:46.653882 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c29cf-c2ad-4f0a-8648-83de379e1570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:46.654326 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c29cf-c2ad-4f0a-8648-83de379e1570 admin admin] PATH_INFO: `/v3/users/643dc4ec13ca4043a8bc9fc310b3ccbe` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:46.656781 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991c29cf-c2ad-4f0a-8648-83de379e1570 admin admin] RBAC: Authorizing `identity:delete_user(user_id=643dc4ec13ca4043a8bc9fc310b3ccbe)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:46.659788 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991c29cf-c2ad-4f0a-8648-83de379e1570 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:46.715799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0358ddd9-ec8f-4a7d-b018-6b36054420ee None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:46.720329 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dafe90-34e2-4dc1-97b1-c6bd7ab2be2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:46.721006 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dafe90-34e2-4dc1-97b1-c6bd7ab2be2b service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:46.721394 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dafe90-34e2-4dc1-97b1-c6bd7ab2be2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:46.721936 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dafe90-34e2-4dc1-97b1-c6bd7ab2be2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:46.733142 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-991c29cf-c2ad-4f0a-8648-83de379e1570 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '643dc4ec13ca4043a8bc9fc310b3ccbe'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:46.733965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-991c29cf-c2ad-4f0a-8648-83de379e1570 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '643dc4ec13ca4043a8bc9fc310b3ccbe'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:46.735112 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dafe90-34e2-4dc1-97b1-c6bd7ab2be2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:46.736937 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dafe90-34e2-4dc1-97b1-c6bd7ab2be2b service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:46.741987 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-991c29cf-c2ad-4f0a-8648-83de379e1570 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '643dc4ec13ca4043a8bc9fc310b3ccbe'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:46.749568 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 956/1929] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:46 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:33:46.757112 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-991c29cf-c2ad-4f0a-8648-83de379e1570 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '643dc4ec13ca4043a8bc9fc310b3ccbe'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:46.757859 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-991c29cf-c2ad-4f0a-8648-83de379e1570 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '643dc4ec13ca4043a8bc9fc310b3ccbe'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:46.784844 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 974/1930] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:33:46 2022] DELETE /identity/v3/users/643dc4ec13ca4043a8bc9fc310b3ccbe => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:46.788917 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dafe90-34e2-4dc1-97b1-c6bd7ab2be2b service cinder] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:46.789382 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b625fa4-d074-40af-800c-26be6366e4bb None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:46.789730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b625fa4-d074-40af-800c-26be6366e4bb None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:46.790014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b625fa4-d074-40af-800c-26be6366e4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:46.804666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dafe90-34e2-4dc1-97b1-c6bd7ab2be2b service cinder] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:46.821688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-1b625fa4-d074-40af-800c-26be6366e4bb None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:46.823689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-1b625fa4-d074-40af-800c-26be6366e4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:46.886870 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe77b62-adfd-4968-bd52-cb76274cf022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:46.887504 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe77b62-adfd-4968-bd52-cb76274cf022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:46.887780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe77b62-adfd-4968-bd52-cb76274cf022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:46.888053 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe77b62-adfd-4968-bd52-cb76274cf022 admin admin] PATH_INFO: `/v3/users/82e1a8c14c3b40f08c889fad3d0d6076` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:46.889207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe77b62-adfd-4968-bd52-cb76274cf022 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82e1a8c14c3b40f08c889fad3d0d6076)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:46.891143 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe77b62-adfd-4968-bd52-cb76274cf022 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:46.896548 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 975/1931] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:46.906629 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b625fa4-d074-40af-800c-26be6366e4bb None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:46.908590 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-1b625fa4-d074-40af-800c-26be6366e4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:46.954444 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-fbe77b62-adfd-4968-bd52-cb76274cf022 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82e1a8c14c3b40f08c889fad3d0d6076'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:46.955120 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-fbe77b62-adfd-4968-bd52-cb76274cf022 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82e1a8c14c3b40f08c889fad3d0d6076'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:46.955736 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-fbe77b62-adfd-4968-bd52-cb76274cf022 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82e1a8c14c3b40f08c889fad3d0d6076'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:46.963678 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-fbe77b62-adfd-4968-bd52-cb76274cf022 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82e1a8c14c3b40f08c889fad3d0d6076'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:46.976679 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b625fa4-d074-40af-800c-26be6366e4bb None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:46.976878 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:46.979376 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-fbe77b62-adfd-4968-bd52-cb76274cf022 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82e1a8c14c3b40f08c889fad3d0d6076'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:46.981636 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7a700-d0ed-41f1-9f13-178c380c55eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:46.982453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7a700-d0ed-41f1-9f13-178c380c55eb service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:46.982731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7a700-d0ed-41f1-9f13-178c380c55eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:46.983220 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7a700-d0ed-41f1-9f13-178c380c55eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:46.985700 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-a1d7a700-d0ed-41f1-9f13-178c380c55eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:46.996743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 957/1932] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:33:46 2022] DELETE /identity/v3/users/82e1a8c14c3b40f08c889fad3d0d6076 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:47.000086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d7a700-d0ed-41f1-9f13-178c380c55eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:47.001386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d7a700-d0ed-41f1-9f13-178c380c55eb service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:47.001532 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d8553-dc51-4299-b5de-993a67038af6 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:47.001886 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d8553-dc51-4299-b5de-993a67038af6 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:47.002602 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d8553-dc51-4299-b5de-993a67038af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:47.014437 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 976/1933] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:33:47.028538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-dc8d8553-dc51-4299-b5de-993a67038af6 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:47.029942 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-dc8d8553-dc51-4299-b5de-993a67038af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:47.082796 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7a700-d0ed-41f1-9f13-178c380c55eb service neutron] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:47.084294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-a1d7a700-d0ed-41f1-9f13-178c380c55eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:47.092043 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:47.108431 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7a700-d0ed-41f1-9f13-178c380c55eb service neutron] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:47.109350 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 958/1934] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:33:47.114082 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351afade-dfcb-48d5-a614-7c0b690fdff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:47.114918 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351afade-dfcb-48d5-a614-7c0b690fdff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:47.115337 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351afade-dfcb-48d5-a614-7c0b690fdff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:47.115613 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351afade-dfcb-48d5-a614-7c0b690fdff1 admin admin] PATH_INFO: `/v3/projects/0a61d050e0304da88702b8a62ba1cbd8` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:47.116742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351afade-dfcb-48d5-a614-7c0b690fdff1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a61d050e0304da88702b8a62ba1cbd8)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:47.118711 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351afade-dfcb-48d5-a614-7c0b690fdff1 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:47.123916 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d8553-dc51-4299-b5de-993a67038af6 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:47.125819 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-dc8d8553-dc51-4299-b5de-993a67038af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:47.178368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-351afade-dfcb-48d5-a614-7c0b690fdff1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a61d050e0304da88702b8a62ba1cbd8', 'request_id': 'req-351afade-dfcb-48d5-a614-7c0b690fdff1'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:47.185995 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-351afade-dfcb-48d5-a614-7c0b690fdff1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a61d050e0304da88702b8a62ba1cbd8', 'request_id': 'req-351afade-dfcb-48d5-a614-7c0b690fdff1'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:47.198441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-351afade-dfcb-48d5-a614-7c0b690fdff1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a61d050e0304da88702b8a62ba1cbd8', 'request_id': 'req-351afade-dfcb-48d5-a614-7c0b690fdff1'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:47.206258 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d8553-dc51-4299-b5de-993a67038af6 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:47.225591 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-351afade-dfcb-48d5-a614-7c0b690fdff1 admin admin] The token cache is being invalidate because project 0a61d050e0304da88702b8a62ba1cbd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:33:47.227836 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-351afade-dfcb-48d5-a614-7c0b690fdff1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:47.229294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-351afade-dfcb-48d5-a614-7c0b690fdff1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:47.232979 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 977/1935] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:33:47 2022] DELETE /identity/v3/projects/0a61d050e0304da88702b8a62ba1cbd8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:33:47.237178 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351afade-dfcb-48d5-a614-7c0b690fdff1 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:47.240291 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-351afade-dfcb-48d5-a614-7c0b690fdff1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:47.331256 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351afade-dfcb-48d5-a614-7c0b690fdff1 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:47.336955 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595578a0-48f7-463c-ad06-d3eb49b2053f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:47.338643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595578a0-48f7-463c-ad06-d3eb49b2053f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:47.339032 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595578a0-48f7-463c-ad06-d3eb49b2053f admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:47.339405 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595578a0-48f7-463c-ad06-d3eb49b2053f admin admin] PATH_INFO: `/v3/users/1d618382fe2b45a9bb4b77d14c04be83` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:47.340958 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595578a0-48f7-463c-ad06-d3eb49b2053f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d618382fe2b45a9bb4b77d14c04be83)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:47.343275 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595578a0-48f7-463c-ad06-d3eb49b2053f admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:47.389833 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5318d9d-3fa6-4e1d-bb65-e4b04861d60b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:47.390491 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5318d9d-3fa6-4e1d-bb65-e4b04861d60b service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:47.390864 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5318d9d-3fa6-4e1d-bb65-e4b04861d60b service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:47.391044 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5318d9d-3fa6-4e1d-bb65-e4b04861d60b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:47.396596 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-f5318d9d-3fa6-4e1d-bb65-e4b04861d60b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:47.412429 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5318d9d-3fa6-4e1d-bb65-e4b04861d60b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:47.413891 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5318d9d-3fa6-4e1d-bb65-e4b04861d60b service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:47.427099 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 959/1936] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:33:47.438383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-595578a0-48f7-463c-ad06-d3eb49b2053f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d618382fe2b45a9bb4b77d14c04be83'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:47.439373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-595578a0-48f7-463c-ad06-d3eb49b2053f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d618382fe2b45a9bb4b77d14c04be83'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:47.453852 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-595578a0-48f7-463c-ad06-d3eb49b2053f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d618382fe2b45a9bb4b77d14c04be83'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:47.473609 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-595578a0-48f7-463c-ad06-d3eb49b2053f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d618382fe2b45a9bb4b77d14c04be83'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:47.474857 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-595578a0-48f7-463c-ad06-d3eb49b2053f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d618382fe2b45a9bb4b77d14c04be83'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:47.510008 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 978/1937] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:33:47 2022] DELETE /identity/v3/users/1d618382fe2b45a9bb4b77d14c04be83 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:47.740143 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5318d9d-3fa6-4e1d-bb65-e4b04861d60b service neutron] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:47.748243 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-f5318d9d-3fa6-4e1d-bb65-e4b04861d60b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:47.770710 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5318d9d-3fa6-4e1d-bb65-e4b04861d60b service neutron] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:47.907129 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:47.940794 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30673f0e-36ea-4f49-88dd-afc22bbc7c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:47.942202 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30673f0e-36ea-4f49-88dd-afc22bbc7c32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:47.942961 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30673f0e-36ea-4f49-88dd-afc22bbc7c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:47.945008 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30673f0e-36ea-4f49-88dd-afc22bbc7c32 admin admin] PATH_INFO: `/v3/projects/a866fb8dd4c04d478a7b5ddb59b018b0` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:47.978770 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30673f0e-36ea-4f49-88dd-afc22bbc7c32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a866fb8dd4c04d478a7b5ddb59b018b0)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:47.980814 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30673f0e-36ea-4f49-88dd-afc22bbc7c32 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:48.114417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-30673f0e-36ea-4f49-88dd-afc22bbc7c32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a866fb8dd4c04d478a7b5ddb59b018b0', 'request_id': 'req-30673f0e-36ea-4f49-88dd-afc22bbc7c32'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:48.124516 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-30673f0e-36ea-4f49-88dd-afc22bbc7c32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a866fb8dd4c04d478a7b5ddb59b018b0', 'request_id': 'req-30673f0e-36ea-4f49-88dd-afc22bbc7c32'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:48.133118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-30673f0e-36ea-4f49-88dd-afc22bbc7c32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a866fb8dd4c04d478a7b5ddb59b018b0', 'request_id': 'req-30673f0e-36ea-4f49-88dd-afc22bbc7c32'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:48.155002 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-30673f0e-36ea-4f49-88dd-afc22bbc7c32 admin admin] The token cache is being invalidate because project a866fb8dd4c04d478a7b5ddb59b018b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:33:48.155501 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-30673f0e-36ea-4f49-88dd-afc22bbc7c32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:48.156337 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-30673f0e-36ea-4f49-88dd-afc22bbc7c32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:48.163782 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 960/1938] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:33:47 2022] DELETE /identity/v3/projects/a866fb8dd4c04d478a7b5ddb59b018b0 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:48.165427 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6029b-a2b7-44ef-9db7-077471bc4a13 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:48.165909 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6029b-a2b7-44ef-9db7-077471bc4a13 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:48.166014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6029b-a2b7-44ef-9db7-077471bc4a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:48.177622 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:48.196464 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-3ee6029b-a2b7-44ef-9db7-077471bc4a13 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:48.197934 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-3ee6029b-a2b7-44ef-9db7-077471bc4a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:48.310238 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 979/1939] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:48.312195 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7d8d1-29f6-452c-8f2f-19a8eb69e195 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:48.312509 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7d8d1-29f6-452c-8f2f-19a8eb69e195 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:48.312744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7d8d1-29f6-452c-8f2f-19a8eb69e195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:48.341495 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6029b-a2b7-44ef-9db7-077471bc4a13 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:48.343149 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-3ee6029b-a2b7-44ef-9db7-077471bc4a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:48.351320 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-73e7d8d1-29f6-452c-8f2f-19a8eb69e195 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:48.352548 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-73e7d8d1-29f6-452c-8f2f-19a8eb69e195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:48.365415 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 961/1940] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:48.366987 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6029b-a2b7-44ef-9db7-077471bc4a13 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:48.369436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e7d8d1-29f6-452c-8f2f-19a8eb69e195 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:48.379522 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36365a31-bc24-4fd7-abe8-7d7d3c370c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:48.380326 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36365a31-bc24-4fd7-abe8-7d7d3c370c31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:48.380594 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36365a31-bc24-4fd7-abe8-7d7d3c370c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:48.380778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36365a31-bc24-4fd7-abe8-7d7d3c370c31 admin admin] PATH_INFO: `/v3/projects/b6fe68fa46be4280aa7633156c101706` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:48.382337 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e7d8d1-29f6-452c-8f2f-19a8eb69e195 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:48.392688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36365a31-bc24-4fd7-abe8-7d7d3c370c31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6fe68fa46be4280aa7633156c101706)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:48.394704 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36365a31-bc24-4fd7-abe8-7d7d3c370c31 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:48.413954 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e319178-f8ea-4f2d-94e3-9f5f4e0fb24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:48.414875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e319178-f8ea-4f2d-94e3-9f5f4e0fb24e admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:48.415371 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e319178-f8ea-4f2d-94e3-9f5f4e0fb24e admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:48.416009 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e319178-f8ea-4f2d-94e3-9f5f4e0fb24e admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:48.417494 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e319178-f8ea-4f2d-94e3-9f5f4e0fb24e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:33:48.418086 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e319178-f8ea-4f2d-94e3-9f5f4e0fb24e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:48.420610 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e319178-f8ea-4f2d-94e3-9f5f4e0fb24e admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:48.486094 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 962/1941] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:33:48.488833 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e319178-f8ea-4f2d-94e3-9f5f4e0fb24e admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:48.491335 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-5e319178-f8ea-4f2d-94e3-9f5f4e0fb24e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:48.495733 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-36365a31-bc24-4fd7-abe8-7d7d3c370c31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6fe68fa46be4280aa7633156c101706', 'request_id': 'req-36365a31-bc24-4fd7-abe8-7d7d3c370c31'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:48.506076 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-36365a31-bc24-4fd7-abe8-7d7d3c370c31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6fe68fa46be4280aa7633156c101706', 'request_id': 'req-36365a31-bc24-4fd7-abe8-7d7d3c370c31'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:48.512775 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e319178-f8ea-4f2d-94e3-9f5f4e0fb24e admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:48.527363 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-36365a31-bc24-4fd7-abe8-7d7d3c370c31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6fe68fa46be4280aa7633156c101706', 'request_id': 'req-36365a31-bc24-4fd7-abe8-7d7d3c370c31'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:48.548828 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-36365a31-bc24-4fd7-abe8-7d7d3c370c31 admin admin] The token cache is being invalidate because project b6fe68fa46be4280aa7633156c101706 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:33:48.549470 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-36365a31-bc24-4fd7-abe8-7d7d3c370c31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:48.550839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-36365a31-bc24-4fd7-abe8-7d7d3c370c31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:48.552922 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 980/1942] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:33:48 2022] DELETE /identity/v3/projects/b6fe68fa46be4280aa7633156c101706 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:33:48.554458 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36365a31-bc24-4fd7-abe8-7d7d3c370c31 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:48.556900 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-36365a31-bc24-4fd7-abe8-7d7d3c370c31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:48.624551 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f131add-4b45-4a21-bd44-1302ee000441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:48.625314 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f131add-4b45-4a21-bd44-1302ee000441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:48.625723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f131add-4b45-4a21-bd44-1302ee000441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:48.626122 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f131add-4b45-4a21-bd44-1302ee000441 admin admin] PATH_INFO: `/v3/projects/f4e295a6d6104d8483f8eea1f9ba3707` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:48.629269 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f131add-4b45-4a21-bd44-1302ee000441 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4e295a6d6104d8483f8eea1f9ba3707)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:48.638084 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f131add-4b45-4a21-bd44-1302ee000441 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:48.680158 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36365a31-bc24-4fd7-abe8-7d7d3c370c31 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:48.687875 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa5856-19fa-410e-b491-7051860bb9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:48.689006 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa5856-19fa-410e-b491-7051860bb9e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:48.689395 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa5856-19fa-410e-b491-7051860bb9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:48.690522 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa5856-19fa-410e-b491-7051860bb9e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:48.692319 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa5856-19fa-410e-b491-7051860bb9e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:48.696680 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa5856-19fa-410e-b491-7051860bb9e3 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:48.704984 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-7f131add-4b45-4a21-bd44-1302ee000441 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4e295a6d6104d8483f8eea1f9ba3707', 'request_id': 'req-7f131add-4b45-4a21-bd44-1302ee000441'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:48.719145 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-7f131add-4b45-4a21-bd44-1302ee000441 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4e295a6d6104d8483f8eea1f9ba3707', 'request_id': 'req-7f131add-4b45-4a21-bd44-1302ee000441'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:48.729015 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:48.732520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-7f131add-4b45-4a21-bd44-1302ee000441 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4e295a6d6104d8483f8eea1f9ba3707', 'request_id': 'req-7f131add-4b45-4a21-bd44-1302ee000441'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:48.750829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 981/1943] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:33:48 2022] POST /identity/v3/projects => generated 389 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:48.751795 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa5856-19fa-410e-b491-7051860bb9e3 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:48.753839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-32fa5856-19fa-410e-b491-7051860bb9e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:48.756979 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-7f131add-4b45-4a21-bd44-1302ee000441 admin admin] The token cache is being invalidate because project f4e295a6d6104d8483f8eea1f9ba3707 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:33:48.757277 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-7f131add-4b45-4a21-bd44-1302ee000441 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:48.757948 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-7f131add-4b45-4a21-bd44-1302ee000441 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:48.766107 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 963/1944] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:33:48 2022] DELETE /identity/v3/projects/f4e295a6d6104d8483f8eea1f9ba3707 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:48.766107 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f131add-4b45-4a21-bd44-1302ee000441 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:48.766107 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-7f131add-4b45-4a21-bd44-1302ee000441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:48.790853 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:48.877909 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f131add-4b45-4a21-bd44-1302ee000441 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:48.887064 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa5856-19fa-410e-b491-7051860bb9e3 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:48.893319 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db75054-9a9b-4878-ba52-0075acc50478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:48.894231 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db75054-9a9b-4878-ba52-0075acc50478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:48.894607 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db75054-9a9b-4878-ba52-0075acc50478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:48.895055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db75054-9a9b-4878-ba52-0075acc50478 admin admin] PATH_INFO: `/v3/users/c258c18f7d2c41d8926f05a11a544902` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:48.896840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db75054-9a9b-4878-ba52-0075acc50478 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c258c18f7d2c41d8926f05a11a544902)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:48.898967 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db75054-9a9b-4878-ba52-0075acc50478 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:48.903970 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785f9e3-c4fa-4893-8e1f-34c456a5ab8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:48.904780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785f9e3-c4fa-4893-8e1f-34c456a5ab8b admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:48.905062 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785f9e3-c4fa-4893-8e1f-34c456a5ab8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:48.905341 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785f9e3-c4fa-4893-8e1f-34c456a5ab8b admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:48.905984 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785f9e3-c4fa-4893-8e1f-34c456a5ab8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:48.911127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785f9e3-c4fa-4893-8e1f-34c456a5ab8b admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:48.962119 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 964/1945] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:33:48 2022] POST /identity/v3/users => generated 295 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:48.962119 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841922b8-dafe-4338-bf5e-b814ee72659c None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:48.962119 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841922b8-dafe-4338-bf5e-b814ee72659c None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:48.962119 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841922b8-dafe-4338-bf5e-b814ee72659c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:48.988975 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-841922b8-dafe-4338-bf5e-b814ee72659c None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:48.990344 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-841922b8-dafe-4338-bf5e-b814ee72659c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:49.001282 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 965/1946] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:49.002297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841922b8-dafe-4338-bf5e-b814ee72659c None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:49.017471 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841922b8-dafe-4338-bf5e-b814ee72659c None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:49.021388 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-4db75054-9a9b-4878-ba52-0075acc50478 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c258c18f7d2c41d8926f05a11a544902'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:49.022643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-4db75054-9a9b-4878-ba52-0075acc50478 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c258c18f7d2c41d8926f05a11a544902'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:49.024784 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501259e-58af-4791-b009-fa3148e0ffbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:49.027173 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501259e-58af-4791-b009-fa3148e0ffbf admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:49.027488 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501259e-58af-4791-b009-fa3148e0ffbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:49.028059 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501259e-58af-4791-b009-fa3148e0ffbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:49.029448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f501259e-58af-4791-b009-fa3148e0ffbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:49.031640 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f501259e-58af-4791-b009-fa3148e0ffbf admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:49.046731 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 966/1947] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:49.049182 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ac25b-ebfc-48a8-8d3a-89e17f6ac0e1 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:49.049182 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ac25b-ebfc-48a8-8d3a-89e17f6ac0e1 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:49.049182 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ac25b-ebfc-48a8-8d3a-89e17f6ac0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:49.052081 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-4db75054-9a9b-4878-ba52-0075acc50478 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c258c18f7d2c41d8926f05a11a544902'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:49.075815 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-424ac25b-ebfc-48a8-8d3a-89e17f6ac0e1 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:49.077433 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-424ac25b-ebfc-48a8-8d3a-89e17f6ac0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:49.090302 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 967/1948] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:49.091644 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ac25b-ebfc-48a8-8d3a-89e17f6ac0e1 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:49.093127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-424ac25b-ebfc-48a8-8d3a-89e17f6ac0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:49.097272 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-4db75054-9a9b-4878-ba52-0075acc50478 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c258c18f7d2c41d8926f05a11a544902'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:49.100909 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-4db75054-9a9b-4878-ba52-0075acc50478 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c258c18f7d2c41d8926f05a11a544902'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:49.110448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ac25b-ebfc-48a8-8d3a-89e17f6ac0e1 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:49.127624 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085994e9-0baa-4c50-b539-a3b605e5f6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:49.128404 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085994e9-0baa-4c50-b539-a3b605e5f6ba admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:49.128742 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085994e9-0baa-4c50-b539-a3b605e5f6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:49.129198 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085994e9-0baa-4c50-b539-a3b605e5f6ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:49.138447 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085994e9-0baa-4c50-b539-a3b605e5f6ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:49.140082 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085994e9-0baa-4c50-b539-a3b605e5f6ba admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:49.163063 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 968/1949] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:49.163753 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085994e9-0baa-4c50-b539-a3b605e5f6ba admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:49.164603 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 982/1950] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:33:48 2022] DELETE /identity/v3/users/c258c18f7d2c41d8926f05a11a544902 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:49.180056 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db75054-9a9b-4878-ba52-0075acc50478 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:49.193733 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085994e9-0baa-4c50-b539-a3b605e5f6ba admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:49.339948 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e566f9-b512-4827-919d-6a6491e0485a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:49.340202 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e566f9-b512-4827-919d-6a6491e0485a admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:49.340344 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e566f9-b512-4827-919d-6a6491e0485a admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:49.340551 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e566f9-b512-4827-919d-6a6491e0485a admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:49.341329 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e566f9-b512-4827-919d-6a6491e0485a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:33:49.341640 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e566f9-b512-4827-919d-6a6491e0485a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:49.343383 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e566f9-b512-4827-919d-6a6491e0485a admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:49.364198 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 969/1951] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:33:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:33:49.368690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e566f9-b512-4827-919d-6a6491e0485a admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:49.376774 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db75054-9a9b-4878-ba52-0075acc50478 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:49.383608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49f1c1-79fd-4175-aa9d-dd0112907e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:49.384420 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49f1c1-79fd-4175-aa9d-dd0112907e0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:49.384723 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49f1c1-79fd-4175-aa9d-dd0112907e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:49.385032 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49f1c1-79fd-4175-aa9d-dd0112907e0e admin admin] PATH_INFO: `/v3/projects/2cb3246131a9452b90c3a537d0be4143/users/5762dd01182d4c2a8856591ef56f7ae7/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:49.397336 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e566f9-b512-4827-919d-6a6491e0485a admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:49.413982 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c49f1c1-79fd-4175-aa9d-dd0112907e0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cb3246131a9452b90c3a537d0be4143, user_id=5762dd01182d4c2a8856591ef56f7ae7, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:49.415406 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0cfb0-bcbc-4e68-b873-894a6ebba24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:49.416088 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0cfb0-bcbc-4e68-b873-894a6ebba24d admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:49.416375 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0cfb0-bcbc-4e68-b873-894a6ebba24d admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:49.416652 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0cfb0-bcbc-4e68-b873-894a6ebba24d admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:49.418176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee0cfb0-bcbc-4e68-b873-894a6ebba24d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:49.418402 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c49f1c1-79fd-4175-aa9d-dd0112907e0e admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:49.420194 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee0cfb0-bcbc-4e68-b873-894a6ebba24d admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:49.441831 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 983/1952] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:49 2022] PUT /identity/v3/projects/2cb3246131a9452b90c3a537d0be4143/users/5762dd01182d4c2a8856591ef56f7ae7/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:49.453108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9f518-7d5e-447d-90cc-f24dc09ac1ab None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:49.453397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9f518-7d5e-447d-90cc-f24dc09ac1ab None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:49.453672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9f518-7d5e-447d-90cc-f24dc09ac1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:49.477057 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 970/1953] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:33:49 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:49.483203 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0cfb0-bcbc-4e68-b873-894a6ebba24d admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:49.513830 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-e9a9f518-7d5e-447d-90cc-f24dc09ac1ab None None] MFA Rules not processed for user `5762dd01182d4c2a8856591ef56f7ae7`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:49.515561 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-e9a9f518-7d5e-447d-90cc-f24dc09ac1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:49.521304 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:49.539140 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:49.628997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 984/1954] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:49.642054 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9f518-7d5e-447d-90cc-f24dc09ac1ab None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:49.644002 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-e9a9f518-7d5e-447d-90cc-f24dc09ac1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:49.686433 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0cfb0-bcbc-4e68-b873-894a6ebba24d admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:49.697795 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beeba47-0046-452c-a983-244b941db8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:49.698931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beeba47-0046-452c-a983-244b941db8e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:49.699240 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beeba47-0046-452c-a983-244b941db8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:49.699598 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beeba47-0046-452c-a983-244b941db8e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:49.700255 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beeba47-0046-452c-a983-244b941db8e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:49.702297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beeba47-0046-452c-a983-244b941db8e0 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:49.738211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 971/1955] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:33:49 2022] POST /identity/v3/users => generated 291 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:33:49.741512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beeba47-0046-452c-a983-244b941db8e0 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:49.741810 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-5beeba47-0046-452c-a983-244b941db8e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:49.756111 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9f518-7d5e-447d-90cc-f24dc09ac1ab None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:49.760073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beeba47-0046-452c-a983-244b941db8e0 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:49.761772 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a5734-7a04-4383-9fb1-6c251a98ffe4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:49.762624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a5734-7a04-4383-9fb1-6c251a98ffe4 service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:49.762905 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a5734-7a04-4383-9fb1-6c251a98ffe4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:49.763212 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a5734-7a04-4383-9fb1-6c251a98ffe4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:49.766996 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e898e9-08b9-4a20-8f3f-d9e4e4c3fb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:49.767650 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e898e9-08b9-4a20-8f3f-d9e4e4c3fb54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:49.768650 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e898e9-08b9-4a20-8f3f-d9e4e4c3fb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:49.768881 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e898e9-08b9-4a20-8f3f-d9e4e4c3fb54 admin admin] PATH_INFO: `/v3/projects/e623ce4832a44325977a7d76f0d40c28` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:49.770195 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e898e9-08b9-4a20-8f3f-d9e4e4c3fb54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e623ce4832a44325977a7d76f0d40c28)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:49.774546 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e898e9-08b9-4a20-8f3f-d9e4e4c3fb54 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:49.775050 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95a5734-7a04-4383-9fb1-6c251a98ffe4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:49.777336 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95a5734-7a04-4383-9fb1-6c251a98ffe4 service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:49.789077 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 985/1956] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:49 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:33:49.790224 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a5734-7a04-4383-9fb1-6c251a98ffe4 service cinder] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:49.809651 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a5734-7a04-4383-9fb1-6c251a98ffe4 service cinder] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:49.816543 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a02997-e0ab-48c3-95a8-868afef6f00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:49.817407 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a02997-e0ab-48c3-95a8-868afef6f00b admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:49.817837 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a02997-e0ab-48c3-95a8-868afef6f00b admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:49.818304 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a02997-e0ab-48c3-95a8-868afef6f00b admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:49.819309 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a02997-e0ab-48c3-95a8-868afef6f00b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:49.821251 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a02997-e0ab-48c3-95a8-868afef6f00b admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:49.834526 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 986/1957] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:49.838431 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a02997-e0ab-48c3-95a8-868afef6f00b admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:49.851093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a02997-e0ab-48c3-95a8-868afef6f00b admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:49.868138 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-81e898e9-08b9-4a20-8f3f-d9e4e4c3fb54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e623ce4832a44325977a7d76f0d40c28', 'request_id': 'req-81e898e9-08b9-4a20-8f3f-d9e4e4c3fb54'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:49.878550 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-81e898e9-08b9-4a20-8f3f-d9e4e4c3fb54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e623ce4832a44325977a7d76f0d40c28', 'request_id': 'req-81e898e9-08b9-4a20-8f3f-d9e4e4c3fb54'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:49.889938 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-81e898e9-08b9-4a20-8f3f-d9e4e4c3fb54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e623ce4832a44325977a7d76f0d40c28', 'request_id': 'req-81e898e9-08b9-4a20-8f3f-d9e4e4c3fb54'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:49.911827 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-81e898e9-08b9-4a20-8f3f-d9e4e4c3fb54 admin admin] The token cache is being invalidate because project e623ce4832a44325977a7d76f0d40c28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:33:49.912835 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-81e898e9-08b9-4a20-8f3f-d9e4e4c3fb54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:49.913834 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-81e898e9-08b9-4a20-8f3f-d9e4e4c3fb54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:49.916414 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 972/1958] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:33:49 2022] DELETE /identity/v3/projects/e623ce4832a44325977a7d76f0d40c28 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:49.922367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58c3a4-0fc8-4190-90d0-8656a84a0f58 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:49.925013 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58c3a4-0fc8-4190-90d0-8656a84a0f58 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:49.926277 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58c3a4-0fc8-4190-90d0-8656a84a0f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:49.951804 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a0c9ac-63d2-4877-80a6-67ed456eeb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:49.953108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0c9ac-63d2-4877-80a6-67ed456eeb01 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:49.953418 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0c9ac-63d2-4877-80a6-67ed456eeb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:49.955061 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0c9ac-63d2-4877-80a6-67ed456eeb01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:49.956188 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a0c9ac-63d2-4877-80a6-67ed456eeb01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:49.958122 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a0c9ac-63d2-4877-80a6-67ed456eeb01 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:49.977796 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 987/1959] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:49.979919 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a0c9ac-63d2-4877-80a6-67ed456eeb01 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:49.982622 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-a3a0c9ac-63d2-4877-80a6-67ed456eeb01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:49.991837 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-6d58c3a4-0fc8-4190-90d0-8656a84a0f58 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:49.993597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-6d58c3a4-0fc8-4190-90d0-8656a84a0f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:50.001741 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a0c9ac-63d2-4877-80a6-67ed456eeb01 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:50.009009 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 973/1960] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:33:50.010558 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1d87e-63c1-49e0-9088-2b1e349a1468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:50.011733 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1d87e-63c1-49e0-9088-2b1e349a1468 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:50.012015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1d87e-63c1-49e0-9088-2b1e349a1468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:50.012292 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1d87e-63c1-49e0-9088-2b1e349a1468 admin admin] PATH_INFO: `/v3/projects/7be930c22bcf4f369a0a1b00e905f5e5/users/8f6ecbaf4b6c48728f702d9278546a63/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:50.017342 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139f8d8-401b-46f4-9a43-b28c75d5f624 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:50.017700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139f8d8-401b-46f4-9a43-b28c75d5f624 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:50.018005 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139f8d8-401b-46f4-9a43-b28c75d5f624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:50.040984 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa1d87e-63c1-49e0-9088-2b1e349a1468 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7be930c22bcf4f369a0a1b00e905f5e5, user_id=8f6ecbaf4b6c48728f702d9278546a63, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:50.045837 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa1d87e-63c1-49e0-9088-2b1e349a1468 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:50.048124 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-b139f8d8-401b-46f4-9a43-b28c75d5f624 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:50.050122 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-b139f8d8-401b-46f4-9a43-b28c75d5f624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:50.063292 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 974/1961] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:50.068496 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139f8d8-401b-46f4-9a43-b28c75d5f624 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:50.075668 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 988/1962] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:49 2022] PUT /identity/v3/projects/7be930c22bcf4f369a0a1b00e905f5e5/users/8f6ecbaf4b6c48728f702d9278546a63/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:50.087639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139f8d8-401b-46f4-9a43-b28c75d5f624 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:50.088243 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ac01e-8d82-4a13-b0f0-974a8400237d None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:50.088559 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ac01e-8d82-4a13-b0f0-974a8400237d None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:50.089451 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ac01e-8d82-4a13-b0f0-974a8400237d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:50.115043 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:50.133220 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-b03ac01e-8d82-4a13-b0f0-974a8400237d None None] MFA Rules not processed for user `8f6ecbaf4b6c48728f702d9278546a63`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:50.140743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-b03ac01e-8d82-4a13-b0f0-974a8400237d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:50.175541 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b733718f-80ab-469d-a2e7-dfb86b1dd270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:50.176173 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b733718f-80ab-469d-a2e7-dfb86b1dd270 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:50.176448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b733718f-80ab-469d-a2e7-dfb86b1dd270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:50.176715 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b733718f-80ab-469d-a2e7-dfb86b1dd270 admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:50.177367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b733718f-80ab-469d-a2e7-dfb86b1dd270 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:33:50.177662 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b733718f-80ab-469d-a2e7-dfb86b1dd270 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:50.179869 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b733718f-80ab-469d-a2e7-dfb86b1dd270 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:50.200863 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 975/1963] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:33:50.206294 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b733718f-80ab-469d-a2e7-dfb86b1dd270 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:50.237783 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b733718f-80ab-469d-a2e7-dfb86b1dd270 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:50.243217 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22fa25-ab54-4d6b-b61b-eb5dcb5330d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:50.243841 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22fa25-ab54-4d6b-b61b-eb5dcb5330d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:50.244119 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22fa25-ab54-4d6b-b61b-eb5dcb5330d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:50.244389 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22fa25-ab54-4d6b-b61b-eb5dcb5330d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:50.245103 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22fa25-ab54-4d6b-b61b-eb5dcb5330d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:50.247123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22fa25-ab54-4d6b-b61b-eb5dcb5330d1 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:50.253619 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:50.256828 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 989/1964] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:50.262371 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ac01e-8d82-4a13-b0f0-974a8400237d None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:50.282852 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ac01e-8d82-4a13-b0f0-974a8400237d None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:50.288606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7297e-6c1c-4134-bbbe-aca9f88076dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:50.289394 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7297e-6c1c-4134-bbbe-aca9f88076dc admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:50.289912 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7297e-6c1c-4134-bbbe-aca9f88076dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:50.290332 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7297e-6c1c-4134-bbbe-aca9f88076dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:50.294125 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7297e-6c1c-4134-bbbe-aca9f88076dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:50.294309 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 976/1965] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:33:50 2022] POST /identity/v3/projects => generated 387 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:50.295998 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22fa25-ab54-4d6b-b61b-eb5dcb5330d1 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:50.299170 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7297e-6c1c-4134-bbbe-aca9f88076dc admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:50.342202 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 990/1966] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:33:50 2022] POST /identity/v3/projects => generated 379 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:50.346249 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7297e-6c1c-4134-bbbe-aca9f88076dc admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:50.397053 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22fa25-ab54-4d6b-b61b-eb5dcb5330d1 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:50.469472 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7297e-6c1c-4134-bbbe-aca9f88076dc admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:50.473507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668da8d2-eac6-404b-b742-8a0bc9c12764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:50.474635 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668da8d2-eac6-404b-b742-8a0bc9c12764 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:50.474980 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668da8d2-eac6-404b-b742-8a0bc9c12764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:50.475316 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668da8d2-eac6-404b-b742-8a0bc9c12764 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:50.476028 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668da8d2-eac6-404b-b742-8a0bc9c12764 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:50.478033 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668da8d2-eac6-404b-b742-8a0bc9c12764 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:50.480639 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39343972-7c4c-4e6c-8c68-b9ab286ce700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:50.481846 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39343972-7c4c-4e6c-8c68-b9ab286ce700 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:50.482431 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39343972-7c4c-4e6c-8c68-b9ab286ce700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:50.482886 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39343972-7c4c-4e6c-8c68-b9ab286ce700 admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:50.484433 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39343972-7c4c-4e6c-8c68-b9ab286ce700 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:50.486959 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39343972-7c4c-4e6c-8c68-b9ab286ce700 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:50.511704 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 977/1967] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:33:50 2022] POST /identity/v3/users => generated 294 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:50.517188 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668da8d2-eac6-404b-b742-8a0bc9c12764 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:50.531630 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668da8d2-eac6-404b-b742-8a0bc9c12764 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:50.539716 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 991/1968] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:33:50 2022] POST /identity/v3/users => generated 296 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:50.542252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39343972-7c4c-4e6c-8c68-b9ab286ce700 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:50.551832 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedbc3ba-69af-4536-876a-7660bf171e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:50.552454 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbc3ba-69af-4536-876a-7660bf171e8a admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:50.552733 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbc3ba-69af-4536-876a-7660bf171e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:50.553171 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbc3ba-69af-4536-876a-7660bf171e8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:50.553648 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedbc3ba-69af-4536-876a-7660bf171e8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:50.555301 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedbc3ba-69af-4536-876a-7660bf171e8a admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:50.564457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39343972-7c4c-4e6c-8c68-b9ab286ce700 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:50.570201 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f0a6c2-45f6-42b0-b3c0-af4fb2807631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:50.571014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0a6c2-45f6-42b0-b3c0-af4fb2807631 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:50.571395 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0a6c2-45f6-42b0-b3c0-af4fb2807631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:50.571769 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0a6c2-45f6-42b0-b3c0-af4fb2807631 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:50.573014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f0a6c2-45f6-42b0-b3c0-af4fb2807631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:50.576984 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f0a6c2-45f6-42b0-b3c0-af4fb2807631 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:50.577386 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 978/1969] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:50.582572 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedbc3ba-69af-4536-876a-7660bf171e8a admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:50.592622 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 992/1970] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:50.597219 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f0a6c2-45f6-42b0-b3c0-af4fb2807631 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:50.600773 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedbc3ba-69af-4536-876a-7660bf171e8a admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:50.609188 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f0a6c2-45f6-42b0-b3c0-af4fb2807631 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:50.611503 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d60bf-ee07-4a87-b23e-bc4c0d312912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:50.612293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d60bf-ee07-4a87-b23e-bc4c0d312912 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:50.612727 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d60bf-ee07-4a87-b23e-bc4c0d312912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:50.613135 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d60bf-ee07-4a87-b23e-bc4c0d312912 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:50.613934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5d60bf-ee07-4a87-b23e-bc4c0d312912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:50.614361 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef343a8-2e23-4549-a883-85e93c5adb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:50.615016 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef343a8-2e23-4549-a883-85e93c5adb83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:50.615301 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef343a8-2e23-4549-a883-85e93c5adb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:50.615601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef343a8-2e23-4549-a883-85e93c5adb83 admin admin] PATH_INFO: `/v3/projects/63f4949754b7430aa8a52a2da948d36f/users/70e560d31cdb4c43b416b8175783304a/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:50.616708 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5d60bf-ee07-4a87-b23e-bc4c0d312912 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:50.629925 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 979/1971] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:33:50.633803 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d60bf-ee07-4a87-b23e-bc4c0d312912 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:50.637496 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef343a8-2e23-4549-a883-85e93c5adb83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63f4949754b7430aa8a52a2da948d36f, user_id=70e560d31cdb4c43b416b8175783304a, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:50.640301 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef343a8-2e23-4549-a883-85e93c5adb83 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:50.645175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d60bf-ee07-4a87-b23e-bc4c0d312912 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:50.650431 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fec9f3-f189-47e9-8415-f4c5433a1cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:50.650982 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fec9f3-f189-47e9-8415-f4c5433a1cc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:50.651200 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fec9f3-f189-47e9-8415-f4c5433a1cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:50.651580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fec9f3-f189-47e9-8415-f4c5433a1cc3 admin admin] PATH_INFO: `/v3/projects/4ca06d599bbd4e1fb0b18fb3b2abc99e/users/29772aaeb43b4386840dee4491fe71d6/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:50.659245 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 993/1972] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:50 2022] PUT /identity/v3/projects/63f4949754b7430aa8a52a2da948d36f/users/70e560d31cdb4c43b416b8175783304a/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:50.660769 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef343a8-2e23-4549-a883-85e93c5adb83 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:50.668419 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fec9f3-f189-47e9-8415-f4c5433a1cc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ca06d599bbd4e1fb0b18fb3b2abc99e, user_id=29772aaeb43b4386840dee4491fe71d6, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:50.671225 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fec9f3-f189-47e9-8415-f4c5433a1cc3 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:50.703951 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 980/1973] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:50 2022] PUT /identity/v3/projects/4ca06d599bbd4e1fb0b18fb3b2abc99e/users/29772aaeb43b4386840dee4491fe71d6/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:33:50.704680 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:50.707492 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c474d-732d-4247-8e98-94fc9894ceb7 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:50.707720 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c474d-732d-4247-8e98-94fc9894ceb7 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:50.708039 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c474d-732d-4247-8e98-94fc9894ceb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:50.741520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef343a8-2e23-4549-a883-85e93c5adb83 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:50.749900 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-cb6c474d-732d-4247-8e98-94fc9894ceb7 None None] MFA Rules not processed for user `29772aaeb43b4386840dee4491fe71d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:50.751198 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-cb6c474d-732d-4247-8e98-94fc9894ceb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:50.787004 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:50.828968 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02312de2-41a6-4309-b53c-7d60ddd098a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:50.829595 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02312de2-41a6-4309-b53c-7d60ddd098a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:50.829909 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02312de2-41a6-4309-b53c-7d60ddd098a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:50.830186 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02312de2-41a6-4309-b53c-7d60ddd098a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:50.830855 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02312de2-41a6-4309-b53c-7d60ddd098a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:50.832381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02312de2-41a6-4309-b53c-7d60ddd098a8 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:50.847568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 994/1974] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:50.849917 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02312de2-41a6-4309-b53c-7d60ddd098a8 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:50.861505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02312de2-41a6-4309-b53c-7d60ddd098a8 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:50.867216 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ae314-f074-414f-87fe-f0bfa04aeec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:50.868408 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ae314-f074-414f-87fe-f0bfa04aeec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:50.868955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 981/1975] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:50.869181 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ae314-f074-414f-87fe-f0bfa04aeec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:50.869181 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ae314-f074-414f-87fe-f0bfa04aeec2 admin admin] PATH_INFO: `/v3/projects/63f4949754b7430aa8a52a2da948d36f/users/70e560d31cdb4c43b416b8175783304a/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:50.869608 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c474d-732d-4247-8e98-94fc9894ceb7 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:50.872014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ae314-f074-414f-87fe-f0bfa04aeec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63f4949754b7430aa8a52a2da948d36f, user_id=70e560d31cdb4c43b416b8175783304a, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:50.874943 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ae314-f074-414f-87fe-f0bfa04aeec2 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:50.886905 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c474d-732d-4247-8e98-94fc9894ceb7 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:50.900300 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 995/1976] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:50 2022] PUT /identity/v3/projects/63f4949754b7430aa8a52a2da948d36f/users/70e560d31cdb4c43b416b8175783304a/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:33:50.901725 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e959ced-609b-45ee-9afe-bb5537e4711d None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:50.901975 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e959ced-609b-45ee-9afe-bb5537e4711d None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:50.902371 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e959ced-609b-45ee-9afe-bb5537e4711d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:50.945296 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-3e959ced-609b-45ee-9afe-bb5537e4711d None None] MFA Rules not processed for user `70e560d31cdb4c43b416b8175783304a`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:50.946707 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-3e959ced-609b-45ee-9afe-bb5537e4711d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:50.969399 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a12f3d-d4b7-4e01-9718-38a9216f806f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:50.970141 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a12f3d-d4b7-4e01-9718-38a9216f806f admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:50.970490 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a12f3d-d4b7-4e01-9718-38a9216f806f admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:50.970831 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a12f3d-d4b7-4e01-9718-38a9216f806f admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:50.971535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a12f3d-d4b7-4e01-9718-38a9216f806f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:50.975120 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a12f3d-d4b7-4e01-9718-38a9216f806f admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:51.026108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 996/1977] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:50 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:33:51.027724 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e959ced-609b-45ee-9afe-bb5537e4711d None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:51.029181 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-3e959ced-609b-45ee-9afe-bb5537e4711d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:51.036852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 982/1978] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:33:50 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:51.037738 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a12f3d-d4b7-4e01-9718-38a9216f806f admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:51.043855 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e959ced-609b-45ee-9afe-bb5537e4711d None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:51.107222 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:51.121565 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a12f3d-d4b7-4e01-9718-38a9216f806f admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:51.123911 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c534ecdb-64db-4807-9a71-e3bb49e611c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:51.124553 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534ecdb-64db-4807-9a71-e3bb49e611c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:51.124970 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534ecdb-64db-4807-9a71-e3bb49e611c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:51.125248 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534ecdb-64db-4807-9a71-e3bb49e611c3 admin admin] PATH_INFO: `/v3/users/d14e900f3aac4c889a401571450118a3` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:51.126478 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c534ecdb-64db-4807-9a71-e3bb49e611c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d14e900f3aac4c889a401571450118a3)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:51.127190 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a72e4-73bd-42b7-8e19-1eecffa5445a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:51.127799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a72e4-73bd-42b7-8e19-1eecffa5445a admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:51.128078 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a72e4-73bd-42b7-8e19-1eecffa5445a admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:51.128350 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a72e4-73bd-42b7-8e19-1eecffa5445a admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:51.128462 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c534ecdb-64db-4807-9a71-e3bb49e611c3 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:51.128984 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a72e4-73bd-42b7-8e19-1eecffa5445a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:51.131109 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a72e4-73bd-42b7-8e19-1eecffa5445a admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:51.159358 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 983/1979] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:33:51 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:51.160116 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a72e4-73bd-42b7-8e19-1eecffa5445a admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:51.163430 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-a90a72e4-73bd-42b7-8e19-1eecffa5445a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:51.184237 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c534ecdb-64db-4807-9a71-e3bb49e611c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd14e900f3aac4c889a401571450118a3'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:51.185108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c534ecdb-64db-4807-9a71-e3bb49e611c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd14e900f3aac4c889a401571450118a3'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:51.194438 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c534ecdb-64db-4807-9a71-e3bb49e611c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd14e900f3aac4c889a401571450118a3'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:51.208917 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c534ecdb-64db-4807-9a71-e3bb49e611c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd14e900f3aac4c889a401571450118a3'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:51.209776 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c534ecdb-64db-4807-9a71-e3bb49e611c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd14e900f3aac4c889a401571450118a3'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:51.218930 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:51.227148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 997/1980] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:33:51 2022] DELETE /identity/v3/users/d14e900f3aac4c889a401571450118a3 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:51.228111 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c534ecdb-64db-4807-9a71-e3bb49e611c3 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:51.229619 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-c534ecdb-64db-4807-9a71-e3bb49e611c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:51.235554 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a72e4-73bd-42b7-8e19-1eecffa5445a admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:51.321887 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493dd2f-67f2-417d-8e96-c0e2d415ae75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:51.322835 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493dd2f-67f2-417d-8e96-c0e2d415ae75 service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:51.323838 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493dd2f-67f2-417d-8e96-c0e2d415ae75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:51.324205 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493dd2f-67f2-417d-8e96-c0e2d415ae75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:51.326254 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-c493dd2f-67f2-417d-8e96-c0e2d415ae75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:51.359949 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c534ecdb-64db-4807-9a71-e3bb49e611c3 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:51.367107 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420ea45-a983-4540-8c66-2ad6515f6a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:51.367732 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420ea45-a983-4540-8c66-2ad6515f6a30 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:51.368014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420ea45-a983-4540-8c66-2ad6515f6a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:51.368401 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420ea45-a983-4540-8c66-2ad6515f6a30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:51.369176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a420ea45-a983-4540-8c66-2ad6515f6a30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:51.371601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a420ea45-a983-4540-8c66-2ad6515f6a30 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:51.386534 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 998/1981] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:51.388415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447dc042-f938-41f5-9c94-7b8b69160037 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:51.388779 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447dc042-f938-41f5-9c94-7b8b69160037 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:51.389127 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447dc042-f938-41f5-9c94-7b8b69160037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:51.415376 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c493dd2f-67f2-417d-8e96-c0e2d415ae75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:51.416456 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-447dc042-f938-41f5-9c94-7b8b69160037 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:51.417243 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c493dd2f-67f2-417d-8e96-c0e2d415ae75 service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:51.417867 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-447dc042-f938-41f5-9c94-7b8b69160037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:51.430504 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 984/1982] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:51 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:33:51.430694 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 999/1983] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:51.430812 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493dd2f-67f2-417d-8e96-c0e2d415ae75 service cinder] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:51.439975 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447dc042-f938-41f5-9c94-7b8b69160037 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:51.441567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-447dc042-f938-41f5-9c94-7b8b69160037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:51.441853 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493dd2f-67f2-417d-8e96-c0e2d415ae75 service cinder] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:51.456798 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7104a07-dcf4-4282-9f2d-b3eeb38310f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:51.458528 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7104a07-dcf4-4282-9f2d-b3eeb38310f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:51.458977 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7104a07-dcf4-4282-9f2d-b3eeb38310f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:51.459393 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7104a07-dcf4-4282-9f2d-b3eeb38310f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:51.460473 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7104a07-dcf4-4282-9f2d-b3eeb38310f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:51.466894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7104a07-dcf4-4282-9f2d-b3eeb38310f8 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:51.482938 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 985/1984] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:51.484676 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7104a07-dcf4-4282-9f2d-b3eeb38310f8 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:51.499606 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7104a07-dcf4-4282-9f2d-b3eeb38310f8 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:51.506377 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83905b2a-5444-461a-ad70-fdd7a0b7efb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:51.507053 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83905b2a-5444-461a-ad70-fdd7a0b7efb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:51.507328 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83905b2a-5444-461a-ad70-fdd7a0b7efb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:51.507598 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83905b2a-5444-461a-ad70-fdd7a0b7efb5 admin admin] PATH_INFO: `/v3/projects/793a2228f8d647779cb892e54164a073/users/6a840e59d47b4e48b3f58fe736647ad0/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:51.526351 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83905b2a-5444-461a-ad70-fdd7a0b7efb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=793a2228f8d647779cb892e54164a073, user_id=6a840e59d47b4e48b3f58fe736647ad0, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:51.529157 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83905b2a-5444-461a-ad70-fdd7a0b7efb5 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:51.532853 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447dc042-f938-41f5-9c94-7b8b69160037 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:51.540460 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a75e5-ef6a-4776-abfb-e67817a2bacd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:51.541293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a75e5-ef6a-4776-abfb-e67817a2bacd service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:51.542543 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a75e5-ef6a-4776-abfb-e67817a2bacd service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:51.544167 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a75e5-ef6a-4776-abfb-e67817a2bacd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:51.546105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 986/1985] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:51 2022] PUT /identity/v3/projects/793a2228f8d647779cb892e54164a073/users/6a840e59d47b4e48b3f58fe736647ad0/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:51.557944 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006fa19-dee4-41d8-b77a-ecde18021b65 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:51.563040 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006fa19-dee4-41d8-b77a-ecde18021b65 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:51.563040 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006fa19-dee4-41d8-b77a-ecde18021b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:51.568154 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005a75e5-ef6a-4776-abfb-e67817a2bacd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:51.570347 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005a75e5-ef6a-4776-abfb-e67817a2bacd service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:51.581276 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:51.589284 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1000/1986] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:33:51.625793 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-1006fa19-dee4-41d8-b77a-ecde18021b65 None None] MFA Rules not processed for user `6a840e59d47b4e48b3f58fe736647ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:51.627467 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-1006fa19-dee4-41d8-b77a-ecde18021b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:51.771561 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 987/1987] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:51.776657 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a75e5-ef6a-4776-abfb-e67817a2bacd service neutron] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:51.794397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a75e5-ef6a-4776-abfb-e67817a2bacd service neutron] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:51.890151 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19db15b-defd-48aa-a1e0-b0142c7116c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:51.892071 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19db15b-defd-48aa-a1e0-b0142c7116c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:51.892545 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19db15b-defd-48aa-a1e0-b0142c7116c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:51.892972 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19db15b-defd-48aa-a1e0-b0142c7116c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:51.893627 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19db15b-defd-48aa-a1e0-b0142c7116c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:51.895591 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19db15b-defd-48aa-a1e0-b0142c7116c4 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:51.944012 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1001/1988] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:33:51 2022] POST /identity/v3/projects => generated 387 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:51.945617 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1006fa19-dee4-41d8-b77a-ecde18021b65 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:51.955458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:51.962711 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19db15b-defd-48aa-a1e0-b0142c7116c4 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:51.965780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1006fa19-dee4-41d8-b77a-ecde18021b65 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:52.055578 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb70e6-e8cf-4edf-8683-7cd5993d240a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:52.056247 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb70e6-e8cf-4edf-8683-7cd5993d240a admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:52.056464 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb70e6-e8cf-4edf-8683-7cd5993d240a admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:52.056736 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb70e6-e8cf-4edf-8683-7cd5993d240a admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:52.057421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb70e6-e8cf-4edf-8683-7cd5993d240a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:52.059513 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb70e6-e8cf-4edf-8683-7cd5993d240a admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:52.075074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19db15b-defd-48aa-a1e0-b0142c7116c4 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:52.081598 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fda76b-9ed7-4e84-81ac-59c91117fbc4 tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70e560d31cdb4c43b416b8175783304a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63f4949754b7430aa8a52a2da948d36f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:52.082274 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fda76b-9ed7-4e84-81ac-59c91117fbc4 tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:52.082617 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fda76b-9ed7-4e84-81ac-59c91117fbc4 tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:52.082957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fda76b-9ed7-4e84-81ac-59c91117fbc4 tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:52.084126 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fda76b-9ed7-4e84-81ac-59c91117fbc4 tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:33:52.084126 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fda76b-9ed7-4e84-81ac-59c91117fbc4 tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:52.085688 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 988/1989] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:33:51 2022] POST /identity/v3/users => generated 300 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:52.085804 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fda76b-9ed7-4e84-81ac-59c91117fbc4 tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:52.090138 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb70e6-e8cf-4edf-8683-7cd5993d240a admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:52.091907 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-62fb70e6-e8cf-4edf-8683-7cd5993d240a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:52.099952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1002/1990] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:33:52.100499 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fda76b-9ed7-4e84-81ac-59c91117fbc4 tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:52.105933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb70e6-e8cf-4edf-8683-7cd5993d240a admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:52.111472 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cfcf2c-bbe4-4a4c-8dd6-d92a0e4ac714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:52.112183 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfcf2c-bbe4-4a4c-8dd6-d92a0e4ac714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:52.112537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfcf2c-bbe4-4a4c-8dd6-d92a0e4ac714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:52.112884 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfcf2c-bbe4-4a4c-8dd6-d92a0e4ac714 admin admin] PATH_INFO: `/v3/projects/9b241373837649d19fc60b7220ae228e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:52.113279 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fda76b-9ed7-4e84-81ac-59c91117fbc4 tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:52.114193 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cfcf2c-bbe4-4a4c-8dd6-d92a0e4ac714 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b241373837649d19fc60b7220ae228e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:52.116100 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cfcf2c-bbe4-4a4c-8dd6-d92a0e4ac714 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:52.119365 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94abdff2-5d67-46a3-92f0-7be3f02b7e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:52.120185 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94abdff2-5d67-46a3-92f0-7be3f02b7e3f admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:52.120616 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94abdff2-5d67-46a3-92f0-7be3f02b7e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:52.121051 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94abdff2-5d67-46a3-92f0-7be3f02b7e3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:52.121869 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94abdff2-5d67-46a3-92f0-7be3f02b7e3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:52.123635 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94abdff2-5d67-46a3-92f0-7be3f02b7e3f admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:52.135354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1003/1991] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:52.136391 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94abdff2-5d67-46a3-92f0-7be3f02b7e3f admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:52.147263 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94abdff2-5d67-46a3-92f0-7be3f02b7e3f admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:52.153648 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f1ba2-f39d-47c1-bb10-a5c2ca81158e tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70e560d31cdb4c43b416b8175783304a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63f4949754b7430aa8a52a2da948d36f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:52.154332 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f1ba2-f39d-47c1-bb10-a5c2ca81158e tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:52.154624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f1ba2-f39d-47c1-bb10-a5c2ca81158e tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:52.154991 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f1ba2-f39d-47c1-bb10-a5c2ca81158e tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:52.155838 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f1ba2-f39d-47c1-bb10-a5c2ca81158e tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:52.157757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f1ba2-f39d-47c1-bb10-a5c2ca81158e tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:52.177877 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-82cfcf2c-bbe4-4a4c-8dd6-d92a0e4ac714 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b241373837649d19fc60b7220ae228e', 'request_id': 'req-82cfcf2c-bbe4-4a4c-8dd6-d92a0e4ac714'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:52.186872 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-82cfcf2c-bbe4-4a4c-8dd6-d92a0e4ac714 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b241373837649d19fc60b7220ae228e', 'request_id': 'req-82cfcf2c-bbe4-4a4c-8dd6-d92a0e4ac714'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:52.187836 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1004/1992] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:33:52 2022] POST /identity/v3/projects => generated 352 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:52.190968 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f1ba2-f39d-47c1-bb10-a5c2ca81158e tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:52.197288 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-82cfcf2c-bbe4-4a4c-8dd6-d92a0e4ac714 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b241373837649d19fc60b7220ae228e', 'request_id': 'req-82cfcf2c-bbe4-4a4c-8dd6-d92a0e4ac714'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:52.211434 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-82cfcf2c-bbe4-4a4c-8dd6-d92a0e4ac714 admin admin] The token cache is being invalidate because project 9b241373837649d19fc60b7220ae228e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:33:52.211858 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-82cfcf2c-bbe4-4a4c-8dd6-d92a0e4ac714 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:52.212947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-82cfcf2c-bbe4-4a4c-8dd6-d92a0e4ac714 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:52.216564 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 989/1993] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:33:52 2022] DELETE /identity/v3/projects/9b241373837649d19fc60b7220ae228e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:52.218939 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cfcf2c-bbe4-4a4c-8dd6-d92a0e4ac714 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:52.220577 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-82cfcf2c-bbe4-4a4c-8dd6-d92a0e4ac714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:52.240771 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:52.264547 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f1ba2-f39d-47c1-bb10-a5c2ca81158e tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:52.271630 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3431a8-4f17-4089-ae4e-87fc488a9e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:52.272531 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3431a8-4f17-4089-ae4e-87fc488a9e8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:52.272890 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3431a8-4f17-4089-ae4e-87fc488a9e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:52.273287 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3431a8-4f17-4089-ae4e-87fc488a9e8b admin admin] PATH_INFO: `/v3/projects/78d9497c839f4002aa630e7ce49e3c34/users/3325b1e1884641c4b036bc4d049f0f9e/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:52.291343 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3431a8-4f17-4089-ae4e-87fc488a9e8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=78d9497c839f4002aa630e7ce49e3c34, user_id=3325b1e1884641c4b036bc4d049f0f9e, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:52.294854 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3431a8-4f17-4089-ae4e-87fc488a9e8b admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:52.296911 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cfcf2c-bbe4-4a4c-8dd6-d92a0e4ac714 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:52.323196 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1005/1994] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:52 2022] PUT /identity/v3/projects/78d9497c839f4002aa630e7ce49e3c34/users/3325b1e1884641c4b036bc4d049f0f9e/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:52.323996 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3431a8-4f17-4089-ae4e-87fc488a9e8b admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:52.325699 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-fc3431a8-4f17-4089-ae4e-87fc488a9e8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:52.326009 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:52.416557 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3431a8-4f17-4089-ae4e-87fc488a9e8b admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:52.420285 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5508b-3ece-46e5-965b-3ce3fad2c70c tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70e560d31cdb4c43b416b8175783304a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63f4949754b7430aa8a52a2da948d36f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:52.420944 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5508b-3ece-46e5-965b-3ce3fad2c70c tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:52.421354 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5508b-3ece-46e5-965b-3ce3fad2c70c tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:52.422063 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5508b-3ece-46e5-965b-3ce3fad2c70c tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:52.422539 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde5508b-3ece-46e5-965b-3ce3fad2c70c tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:33:52.422670 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde5508b-3ece-46e5-965b-3ce3fad2c70c tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:52.424225 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde5508b-3ece-46e5-965b-3ce3fad2c70c tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:52.425842 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee42b87-2c26-462e-add0-06d6cd64421a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:52.426734 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee42b87-2c26-462e-add0-06d6cd64421a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:52.426734 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee42b87-2c26-462e-add0-06d6cd64421a admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:52.427016 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee42b87-2c26-462e-add0-06d6cd64421a admin admin] PATH_INFO: `/v3/users/8f40290cab2147fc83b46bc70401a669` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:52.428168 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee42b87-2c26-462e-add0-06d6cd64421a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f40290cab2147fc83b46bc70401a669)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:52.431275 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee42b87-2c26-462e-add0-06d6cd64421a admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:52.459513 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 990/1995] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:33:52.460500 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5508b-3ece-46e5-965b-3ce3fad2c70c tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:52.466140 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-cde5508b-3ece-46e5-965b-3ce3fad2c70c tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:52.482547 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5508b-3ece-46e5-965b-3ce3fad2c70c tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:52.488272 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42caab-904f-4033-b5ac-62c0d2964b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:52.488887 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42caab-904f-4033-b5ac-62c0d2964b3f admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:52.489257 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42caab-904f-4033-b5ac-62c0d2964b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:52.489672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42caab-904f-4033-b5ac-62c0d2964b3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:52.490830 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42caab-904f-4033-b5ac-62c0d2964b3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:52.492803 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42caab-904f-4033-b5ac-62c0d2964b3f admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:52.519249 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 991/1996] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:52.520316 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42caab-904f-4033-b5ac-62c0d2964b3f admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:52.533993 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-aee42b87-2c26-462e-add0-06d6cd64421a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f40290cab2147fc83b46bc70401a669'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:52.534850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-aee42b87-2c26-462e-add0-06d6cd64421a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f40290cab2147fc83b46bc70401a669'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:52.542986 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-aee42b87-2c26-462e-add0-06d6cd64421a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f40290cab2147fc83b46bc70401a669'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:52.555967 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-aee42b87-2c26-462e-add0-06d6cd64421a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f40290cab2147fc83b46bc70401a669'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:52.557019 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-aee42b87-2c26-462e-add0-06d6cd64421a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f40290cab2147fc83b46bc70401a669'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:52.572867 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1006/1997] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:33:52 2022] DELETE /identity/v3/users/8f40290cab2147fc83b46bc70401a669 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:52.574311 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee42b87-2c26-462e-add0-06d6cd64421a admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:52.575749 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-aee42b87-2c26-462e-add0-06d6cd64421a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:52.603809 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42caab-904f-4033-b5ac-62c0d2964b3f admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:52.647048 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:52.671739 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee42b87-2c26-462e-add0-06d6cd64421a admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:52.698875 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384c5cd-6d49-4fa3-85c7-c90d6b394693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:52.699819 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384c5cd-6d49-4fa3-85c7-c90d6b394693 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:52.699929 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384c5cd-6d49-4fa3-85c7-c90d6b394693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:52.703391 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384c5cd-6d49-4fa3-85c7-c90d6b394693 admin admin] PATH_INFO: `/v3/projects/78d9497c839f4002aa630e7ce49e3c34/users/3325b1e1884641c4b036bc4d049f0f9e/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:52.709729 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384c5cd-6d49-4fa3-85c7-c90d6b394693 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78d9497c839f4002aa630e7ce49e3c34, user_id=3325b1e1884641c4b036bc4d049f0f9e, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:52.719396 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384c5cd-6d49-4fa3-85c7-c90d6b394693 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:52.730307 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc34d6-35d1-48c8-a0f2-d4a3866edfd1 tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70e560d31cdb4c43b416b8175783304a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63f4949754b7430aa8a52a2da948d36f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:52.731155 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc34d6-35d1-48c8-a0f2-d4a3866edfd1 tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:52.731892 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc34d6-35d1-48c8-a0f2-d4a3866edfd1 tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:52.732222 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc34d6-35d1-48c8-a0f2-d4a3866edfd1 tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] PATH_INFO: `/v3/projects/a5dbb3ef06a748f8a02f423900815b3b` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:52.733631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc34d6-35d1-48c8-a0f2-d4a3866edfd1 tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a5dbb3ef06a748f8a02f423900815b3b)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:52.736850 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc34d6-35d1-48c8-a0f2-d4a3866edfd1 tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:52.743298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1007/1998] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:52 2022] PUT /identity/v3/projects/78d9497c839f4002aa630e7ce49e3c34/users/3325b1e1884641c4b036bc4d049f0f9e/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:52.753395 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde9637-7268-44af-82a3-00a207593850 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:52.753703 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde9637-7268-44af-82a3-00a207593850 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:52.753987 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde9637-7268-44af-82a3-00a207593850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:52.798972 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-fcde9637-7268-44af-82a3-00a207593850 None None] MFA Rules not processed for user `3325b1e1884641c4b036bc4d049f0f9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:52.804287 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-fcde9637-7268-44af-82a3-00a207593850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:52.832612 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-39bc34d6-35d1-48c8-a0f2-d4a3866edfd1 tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5dbb3ef06a748f8a02f423900815b3b', 'request_id': 'req-39bc34d6-35d1-48c8-a0f2-d4a3866edfd1'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:52.840853 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-39bc34d6-35d1-48c8-a0f2-d4a3866edfd1 tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5dbb3ef06a748f8a02f423900815b3b', 'request_id': 'req-39bc34d6-35d1-48c8-a0f2-d4a3866edfd1'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:52.842299 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:52.848786 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-39bc34d6-35d1-48c8-a0f2-d4a3866edfd1 tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5dbb3ef06a748f8a02f423900815b3b', 'request_id': 'req-39bc34d6-35d1-48c8-a0f2-d4a3866edfd1'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:52.867636 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-39bc34d6-35d1-48c8-a0f2-d4a3866edfd1 tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] The token cache is being invalidate because project a5dbb3ef06a748f8a02f423900815b3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:33:52.868683 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-39bc34d6-35d1-48c8-a0f2-d4a3866edfd1 tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:52.869578 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-39bc34d6-35d1-48c8-a0f2-d4a3866edfd1 tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:52.874881 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 992/1999] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:33:52 2022] DELETE /identity/v3/projects/a5dbb3ef06a748f8a02f423900815b3b => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:52.919204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1008/2000] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:52 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:33:52.931144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc34d6-35d1-48c8-a0f2-d4a3866edfd1 tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:52.933265 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-39bc34d6-35d1-48c8-a0f2-d4a3866edfd1 tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:53.024594 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde9637-7268-44af-82a3-00a207593850 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:53.026548 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-fcde9637-7268-44af-82a3-00a207593850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:53.027291 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc34d6-35d1-48c8-a0f2-d4a3866edfd1 tempest-VolumeQuotaClassesTest-440919589 tempest-VolumeQuotaClassesTest-440919589-project-admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:53.035133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9d161-f607-4fa1-9ada-be78d026085c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:53.035874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9d161-f607-4fa1-9ada-be78d026085c service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:53.036205 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9d161-f607-4fa1-9ada-be78d026085c service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:53.036543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9d161-f607-4fa1-9ada-be78d026085c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:53.048746 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9d161-f607-4fa1-9ada-be78d026085c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:53.051086 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9d161-f607-4fa1-9ada-be78d026085c service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:53.071452 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 993/2001] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:52 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:33:53.127934 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde9637-7268-44af-82a3-00a207593850 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:53.134916 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d834af5-613c-43cd-98fb-3f33e36c2c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:53.136451 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d834af5-613c-43cd-98fb-3f33e36c2c45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:53.136884 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d834af5-613c-43cd-98fb-3f33e36c2c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:53.137347 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d834af5-613c-43cd-98fb-3f33e36c2c45 admin admin] PATH_INFO: `/v3/projects/5b434968ce3f4982a0c05ba45593b3b9` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:53.139090 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d834af5-613c-43cd-98fb-3f33e36c2c45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b434968ce3f4982a0c05ba45593b3b9)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:53.141063 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d834af5-613c-43cd-98fb-3f33e36c2c45 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:53.180359 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9d161-f607-4fa1-9ada-be78d026085c service cinder] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:53.182169 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-57c9d161-f607-4fa1-9ada-be78d026085c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:53.198272 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9d161-f607-4fa1-9ada-be78d026085c service cinder] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:53.212990 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-0d834af5-613c-43cd-98fb-3f33e36c2c45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b434968ce3f4982a0c05ba45593b3b9', 'request_id': 'req-0d834af5-613c-43cd-98fb-3f33e36c2c45'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:53.220270 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-0d834af5-613c-43cd-98fb-3f33e36c2c45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b434968ce3f4982a0c05ba45593b3b9', 'request_id': 'req-0d834af5-613c-43cd-98fb-3f33e36c2c45'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:53.230402 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-0d834af5-613c-43cd-98fb-3f33e36c2c45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b434968ce3f4982a0c05ba45593b3b9', 'request_id': 'req-0d834af5-613c-43cd-98fb-3f33e36c2c45'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:53.235810 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:53.243300 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-0d834af5-613c-43cd-98fb-3f33e36c2c45 admin admin] The token cache is being invalidate because project 5b434968ce3f4982a0c05ba45593b3b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:33:53.243606 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-0d834af5-613c-43cd-98fb-3f33e36c2c45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:53.245719 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-0d834af5-613c-43cd-98fb-3f33e36c2c45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:53.250955 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1009/2002] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:33:53 2022] DELETE /identity/v3/projects/5b434968ce3f4982a0c05ba45593b3b9 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:53.252757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278139e5-611b-4998-9f6d-acfc36b155f4 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:53.253321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278139e5-611b-4998-9f6d-acfc36b155f4 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:53.254309 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278139e5-611b-4998-9f6d-acfc36b155f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:53.267373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da045ed-bcad-474b-9131-b06f5c02bc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:53.268020 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da045ed-bcad-474b-9131-b06f5c02bc00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:53.268296 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da045ed-bcad-474b-9131-b06f5c02bc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:53.268575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da045ed-bcad-474b-9131-b06f5c02bc00 admin admin] PATH_INFO: `/v3/users/8f6ecbaf4b6c48728f702d9278546a63` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:53.269737 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da045ed-bcad-474b-9131-b06f5c02bc00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f6ecbaf4b6c48728f702d9278546a63)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:53.271763 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da045ed-bcad-474b-9131-b06f5c02bc00 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:53.277739 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-278139e5-611b-4998-9f6d-acfc36b155f4 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:53.279361 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-278139e5-611b-4998-9f6d-acfc36b155f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:53.298587 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1010/2003] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:33:53.299918 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8594357-c5f2-446b-b3f7-e6e9d41d03a0 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:53.300149 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8594357-c5f2-446b-b3f7-e6e9d41d03a0 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:53.300425 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8594357-c5f2-446b-b3f7-e6e9d41d03a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:53.324070 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-a8594357-c5f2-446b-b3f7-e6e9d41d03a0 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:53.325975 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-a8594357-c5f2-446b-b3f7-e6e9d41d03a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:53.330893 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-7da045ed-bcad-474b-9131-b06f5c02bc00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f6ecbaf4b6c48728f702d9278546a63'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:53.331583 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-7da045ed-bcad-474b-9131-b06f5c02bc00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f6ecbaf4b6c48728f702d9278546a63'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:53.332212 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-7da045ed-bcad-474b-9131-b06f5c02bc00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f6ecbaf4b6c48728f702d9278546a63'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:53.342297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1011/2004] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:33:53.343582 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8594357-c5f2-446b-b3f7-e6e9d41d03a0 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:53.345093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-7da045ed-bcad-474b-9131-b06f5c02bc00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f6ecbaf4b6c48728f702d9278546a63'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:53.353882 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8594357-c5f2-446b-b3f7-e6e9d41d03a0 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:53.359290 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2741d8-ed8d-4d49-acf1-883bb13f3866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:53.359464 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-7da045ed-bcad-474b-9131-b06f5c02bc00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f6ecbaf4b6c48728f702d9278546a63'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:53.359818 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2741d8-ed8d-4d49-acf1-883bb13f3866 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:53.360146 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2741d8-ed8d-4d49-acf1-883bb13f3866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:53.360359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2741d8-ed8d-4d49-acf1-883bb13f3866 admin admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:53.361000 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2741d8-ed8d-4d49-acf1-883bb13f3866 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:33:53.361467 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2741d8-ed8d-4d49-acf1-883bb13f3866 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:53.363104 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2741d8-ed8d-4d49-acf1-883bb13f3866 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:53.376430 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1012/2005] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:33:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:33:53.381009 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2741d8-ed8d-4d49-acf1-883bb13f3866 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:53.385533 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 994/2006] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:33:53 2022] DELETE /identity/v3/users/8f6ecbaf4b6c48728f702d9278546a63 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:53.389772 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2956a9-2873-428a-8dda-37e8932c26bc None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:53.390099 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2956a9-2873-428a-8dda-37e8932c26bc None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:53.390381 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2956a9-2873-428a-8dda-37e8932c26bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:53.392260 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2741d8-ed8d-4d49-acf1-883bb13f3866 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:53.399304 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:53.412673 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-4b2956a9-2873-428a-8dda-37e8932c26bc None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:53.414755 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-4b2956a9-2873-428a-8dda-37e8932c26bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:53.462797 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e87319-75e1-4a49-b870-4240b46b65f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:53.463881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e87319-75e1-4a49-b870-4240b46b65f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:53.464210 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e87319-75e1-4a49-b870-4240b46b65f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:53.464654 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e87319-75e1-4a49-b870-4240b46b65f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:53.465518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e87319-75e1-4a49-b870-4240b46b65f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:53.467619 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e87319-75e1-4a49-b870-4240b46b65f8 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:53.485190 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 995/2007] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:53.497538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2956a9-2873-428a-8dda-37e8932c26bc None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:53.501333 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-4b2956a9-2873-428a-8dda-37e8932c26bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:53.509633 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1013/2008] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:33:53 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:33:53.514125 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e87319-75e1-4a49-b870-4240b46b65f8 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:53.582779 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2956a9-2873-428a-8dda-37e8932c26bc None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:53.608045 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e87319-75e1-4a49-b870-4240b46b65f8 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:53.621419 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c216e5c-e1f6-4423-9d0a-13cf9bbe5c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:53.622452 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c216e5c-e1f6-4423-9d0a-13cf9bbe5c18 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:53.622897 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c216e5c-e1f6-4423-9d0a-13cf9bbe5c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:53.623354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c216e5c-e1f6-4423-9d0a-13cf9bbe5c18 admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:53.625132 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c216e5c-e1f6-4423-9d0a-13cf9bbe5c18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:53.627559 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c216e5c-e1f6-4423-9d0a-13cf9bbe5c18 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:53.648631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6adc6-772b-4930-bd5a-7ae0f8870b0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:53.649254 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6adc6-772b-4930-bd5a-7ae0f8870b0d service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:53.649599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6adc6-772b-4930-bd5a-7ae0f8870b0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:53.649877 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6adc6-772b-4930-bd5a-7ae0f8870b0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:53.661540 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e6adc6-772b-4930-bd5a-7ae0f8870b0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:53.663245 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1014/2009] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:33:53 2022] POST /identity/v3/users => generated 292 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:53.666804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e6adc6-772b-4930-bd5a-7ae0f8870b0d service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:53.672169 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c216e5c-e1f6-4423-9d0a-13cf9bbe5c18 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:53.684855 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c216e5c-e1f6-4423-9d0a-13cf9bbe5c18 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:53.686500 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 996/2010] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:33:53.689989 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9662b55-0806-48f6-976d-336e3698c7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:53.690832 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9662b55-0806-48f6-976d-336e3698c7c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:53.691389 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9662b55-0806-48f6-976d-336e3698c7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:53.691766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9662b55-0806-48f6-976d-336e3698c7c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:53.692525 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9662b55-0806-48f6-976d-336e3698c7c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:53.694400 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9662b55-0806-48f6-976d-336e3698c7c6 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:53.716491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1015/2011] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:33:53.717802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6adc6-772b-4930-bd5a-7ae0f8870b0d service neutron] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:53.735087 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6adc6-772b-4930-bd5a-7ae0f8870b0d service neutron] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:53.743921 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb21f2-2f0d-4f33-bb19-78c4af95f14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:53.744843 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb21f2-2f0d-4f33-bb19-78c4af95f14f admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:53.745195 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb21f2-2f0d-4f33-bb19-78c4af95f14f admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:53.745319 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9662b55-0806-48f6-976d-336e3698c7c6 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:53.745408 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb21f2-2f0d-4f33-bb19-78c4af95f14f admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:53.746313 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb21f2-2f0d-4f33-bb19-78c4af95f14f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:53.748206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb21f2-2f0d-4f33-bb19-78c4af95f14f admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:53.751013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-d9662b55-0806-48f6-976d-336e3698c7c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:53.762718 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 997/2012] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:53.767336 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb21f2-2f0d-4f33-bb19-78c4af95f14f admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:53.780670 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb21f2-2f0d-4f33-bb19-78c4af95f14f admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:53.787749 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e84981c-ba58-42c7-a65a-4ea1d77ad522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:53.788977 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84981c-ba58-42c7-a65a-4ea1d77ad522 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:53.789241 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84981c-ba58-42c7-a65a-4ea1d77ad522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:53.789555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84981c-ba58-42c7-a65a-4ea1d77ad522 admin admin] PATH_INFO: `/v3/projects/2510643511c4445eaced89475e51e2b6/users/8ff56cf1087248eeb7d8fcdde52967d4/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:53.793771 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:53.816826 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e84981c-ba58-42c7-a65a-4ea1d77ad522 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2510643511c4445eaced89475e51e2b6, user_id=8ff56cf1087248eeb7d8fcdde52967d4, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:53.819968 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e84981c-ba58-42c7-a65a-4ea1d77ad522 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:53.823969 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9662b55-0806-48f6-976d-336e3698c7c6 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:53.829641 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600e54c-2993-441a-84d9-360c9567616a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:53.830340 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600e54c-2993-441a-84d9-360c9567616a service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:53.830601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600e54c-2993-441a-84d9-360c9567616a service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:53.831248 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600e54c-2993-441a-84d9-360c9567616a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:53.832680 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-7600e54c-2993-441a-84d9-360c9567616a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:53.839278 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 998/2013] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:53 2022] PUT /identity/v3/projects/2510643511c4445eaced89475e51e2b6/users/8ff56cf1087248eeb7d8fcdde52967d4/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:53.846675 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7944aa-713f-4021-a603-52283a505dc8 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:53.847746 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7944aa-713f-4021-a603-52283a505dc8 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:53.847746 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7944aa-713f-4021-a603-52283a505dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:53.897387 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-ad7944aa-713f-4021-a603-52283a505dc8 None None] MFA Rules not processed for user `8ff56cf1087248eeb7d8fcdde52967d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:53.898699 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-ad7944aa-713f-4021-a603-52283a505dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:53.904242 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7600e54c-2993-441a-84d9-360c9567616a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:53.905901 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7600e54c-2993-441a-84d9-360c9567616a service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:53.916698 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1016/2014] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:53 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:33:53.920620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:53.980966 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 999/2015] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:33:53.994807 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600e54c-2993-441a-84d9-360c9567616a service cinder] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:53.996389 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-7600e54c-2993-441a-84d9-360c9567616a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:54.012900 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600e54c-2993-441a-84d9-360c9567616a service cinder] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:54.027336 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7944aa-713f-4021-a603-52283a505dc8 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:54.030963 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-ad7944aa-713f-4021-a603-52283a505dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:54.083428 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0517a-ee3f-4790-8754-9bac646f6e0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:54.084069 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0517a-ee3f-4790-8754-9bac646f6e0c service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:54.084350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0517a-ee3f-4790-8754-9bac646f6e0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:54.084631 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0517a-ee3f-4790-8754-9bac646f6e0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:54.094873 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0517a-ee3f-4790-8754-9bac646f6e0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:54.096396 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0517a-ee3f-4790-8754-9bac646f6e0c service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:54.107103 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1017/2016] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:33:54.127455 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7944aa-713f-4021-a603-52283a505dc8 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:54.133251 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86609ae-8675-4140-b5c9-24b0cbf43c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:54.134055 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86609ae-8675-4140-b5c9-24b0cbf43c62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:54.134455 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86609ae-8675-4140-b5c9-24b0cbf43c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:54.134791 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86609ae-8675-4140-b5c9-24b0cbf43c62 admin admin] PATH_INFO: `/v3/projects/7be930c22bcf4f369a0a1b00e905f5e5` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:54.136542 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86609ae-8675-4140-b5c9-24b0cbf43c62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7be930c22bcf4f369a0a1b00e905f5e5)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:54.138639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86609ae-8675-4140-b5c9-24b0cbf43c62 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:54.195353 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e86609ae-8675-4140-b5c9-24b0cbf43c62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7be930c22bcf4f369a0a1b00e905f5e5', 'request_id': 'req-e86609ae-8675-4140-b5c9-24b0cbf43c62'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:54.211173 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e86609ae-8675-4140-b5c9-24b0cbf43c62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7be930c22bcf4f369a0a1b00e905f5e5', 'request_id': 'req-e86609ae-8675-4140-b5c9-24b0cbf43c62'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:54.229653 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e86609ae-8675-4140-b5c9-24b0cbf43c62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7be930c22bcf4f369a0a1b00e905f5e5', 'request_id': 'req-e86609ae-8675-4140-b5c9-24b0cbf43c62'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:54.248987 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e86609ae-8675-4140-b5c9-24b0cbf43c62 admin admin] The token cache is being invalidate because project 7be930c22bcf4f369a0a1b00e905f5e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:33:54.250254 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e86609ae-8675-4140-b5c9-24b0cbf43c62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:54.251001 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e86609ae-8675-4140-b5c9-24b0cbf43c62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:54.252631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1000/2017] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:33:54 2022] DELETE /identity/v3/projects/7be930c22bcf4f369a0a1b00e905f5e5 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:33:54.256758 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0517a-ee3f-4790-8754-9bac646f6e0c service cinder] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:54.272837 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0517a-ee3f-4790-8754-9bac646f6e0c service cinder] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:54.356836 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6213c2f-6a0c-47c4-9515-d04e543aa512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:54.357477 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6213c2f-6a0c-47c4-9515-d04e543aa512 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:54.357760 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6213c2f-6a0c-47c4-9515-d04e543aa512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:54.358062 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6213c2f-6a0c-47c4-9515-d04e543aa512 admin admin] PATH_INFO: `/v3/users/70e560d31cdb4c43b416b8175783304a` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:54.359240 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6213c2f-6a0c-47c4-9515-d04e543aa512 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70e560d31cdb4c43b416b8175783304a)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:54.361250 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6213c2f-6a0c-47c4-9515-d04e543aa512 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:54.398611 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:54.413690 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d6213c2f-6a0c-47c4-9515-d04e543aa512 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70e560d31cdb4c43b416b8175783304a'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:54.414518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d6213c2f-6a0c-47c4-9515-d04e543aa512 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70e560d31cdb4c43b416b8175783304a'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:54.422789 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d6213c2f-6a0c-47c4-9515-d04e543aa512 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70e560d31cdb4c43b416b8175783304a'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:54.436506 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d6213c2f-6a0c-47c4-9515-d04e543aa512 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70e560d31cdb4c43b416b8175783304a'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:54.437311 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d6213c2f-6a0c-47c4-9515-d04e543aa512 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70e560d31cdb4c43b416b8175783304a'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:54.454187 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1018/2018] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:33:54 2022] DELETE /identity/v3/users/70e560d31cdb4c43b416b8175783304a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:33:54.727793 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86609ae-8675-4140-b5c9-24b0cbf43c62 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:54.729755 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-e86609ae-8675-4140-b5c9-24b0cbf43c62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:54.743224 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86609ae-8675-4140-b5c9-24b0cbf43c62 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:54.798131 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:54.804878 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4253d3e-ac68-47b2-8312-9a3ea74f4888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:54.805517 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4253d3e-ac68-47b2-8312-9a3ea74f4888 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:54.805802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4253d3e-ac68-47b2-8312-9a3ea74f4888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:54.806457 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4253d3e-ac68-47b2-8312-9a3ea74f4888 admin admin] PATH_INFO: `/v3/projects/63f4949754b7430aa8a52a2da948d36f` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:54.808820 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4253d3e-ac68-47b2-8312-9a3ea74f4888 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63f4949754b7430aa8a52a2da948d36f)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:54.810556 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4253d3e-ac68-47b2-8312-9a3ea74f4888 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:54.860907 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-f4253d3e-ac68-47b2-8312-9a3ea74f4888 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63f4949754b7430aa8a52a2da948d36f', 'request_id': 'req-f4253d3e-ac68-47b2-8312-9a3ea74f4888'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:54.867344 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-f4253d3e-ac68-47b2-8312-9a3ea74f4888 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63f4949754b7430aa8a52a2da948d36f', 'request_id': 'req-f4253d3e-ac68-47b2-8312-9a3ea74f4888'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:54.873603 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-f4253d3e-ac68-47b2-8312-9a3ea74f4888 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63f4949754b7430aa8a52a2da948d36f', 'request_id': 'req-f4253d3e-ac68-47b2-8312-9a3ea74f4888'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:54.886937 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-f4253d3e-ac68-47b2-8312-9a3ea74f4888 admin admin] The token cache is being invalidate because project 63f4949754b7430aa8a52a2da948d36f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:33:54.887225 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-f4253d3e-ac68-47b2-8312-9a3ea74f4888 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:54.887890 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-f4253d3e-ac68-47b2-8312-9a3ea74f4888 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:54.889720 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1001/2019] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:33:54 2022] DELETE /identity/v3/projects/63f4949754b7430aa8a52a2da948d36f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:54.895027 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ec695-b9c7-44da-b282-cc4e28351a18 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:54.895500 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ec695-b9c7-44da-b282-cc4e28351a18 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:54.895611 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ec695-b9c7-44da-b282-cc4e28351a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:54.919089 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-cd7ec695-b9c7-44da-b282-cc4e28351a18 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:54.920374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-cd7ec695-b9c7-44da-b282-cc4e28351a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:54.978690 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1019/2020] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:54.986918 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f2f87-7766-4570-97c9-58e3cfc6d6c2 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:54.987176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f2f87-7766-4570-97c9-58e3cfc6d6c2 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:54.987447 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f2f87-7766-4570-97c9-58e3cfc6d6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:55.007428 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-be8f2f87-7766-4570-97c9-58e3cfc6d6c2 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:55.008643 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-be8f2f87-7766-4570-97c9-58e3cfc6d6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:55.016464 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1002/2021] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:55.020539 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ec695-b9c7-44da-b282-cc4e28351a18 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:55.030580 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ec695-b9c7-44da-b282-cc4e28351a18 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:55.035052 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54605f64-ab59-47f5-8611-c1f4acebb39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:55.035639 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54605f64-ab59-47f5-8611-c1f4acebb39a admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:55.035914 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54605f64-ab59-47f5-8611-c1f4acebb39a admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:55.036184 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54605f64-ab59-47f5-8611-c1f4acebb39a admin admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:55.036822 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54605f64-ab59-47f5-8611-c1f4acebb39a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:33:55.037118 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54605f64-ab59-47f5-8611-c1f4acebb39a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:55.038742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54605f64-ab59-47f5-8611-c1f4acebb39a admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:55.051048 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1020/2022] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:33:55.055076 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f2f87-7766-4570-97c9-58e3cfc6d6c2 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:55.064190 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f2f87-7766-4570-97c9-58e3cfc6d6c2 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:55.068769 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054392d-2644-473c-a7a1-fc0ca04e2dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:55.069352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054392d-2644-473c-a7a1-fc0ca04e2dcd admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:55.069625 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054392d-2644-473c-a7a1-fc0ca04e2dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:55.069894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054392d-2644-473c-a7a1-fc0ca04e2dcd admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:55.070575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5054392d-2644-473c-a7a1-fc0ca04e2dcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:55.072464 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5054392d-2644-473c-a7a1-fc0ca04e2dcd admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:55.099568 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1003/2023] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:33:55 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:55.103736 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54605f64-ab59-47f5-8611-c1f4acebb39a admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:55.114232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54605f64-ab59-47f5-8611-c1f4acebb39a admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:55.171813 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47223c5b-3475-4f2c-8f32-3f3d0136f703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:55.172412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47223c5b-3475-4f2c-8f32-3f3d0136f703 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:55.172690 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47223c5b-3475-4f2c-8f32-3f3d0136f703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:55.172961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47223c5b-3475-4f2c-8f32-3f3d0136f703 admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:55.173584 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47223c5b-3475-4f2c-8f32-3f3d0136f703 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:55.175511 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47223c5b-3475-4f2c-8f32-3f3d0136f703 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:55.195973 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1021/2024] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:33:55 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:55.200056 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054392d-2644-473c-a7a1-fc0ca04e2dcd admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:55.210204 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054392d-2644-473c-a7a1-fc0ca04e2dcd admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:55.215252 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775c042-0264-4c82-ac3a-4b245578b23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:55.215837 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775c042-0264-4c82-ac3a-4b245578b23c admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:55.216109 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775c042-0264-4c82-ac3a-4b245578b23c admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:55.216372 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775c042-0264-4c82-ac3a-4b245578b23c admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:55.216994 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775c042-0264-4c82-ac3a-4b245578b23c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:55.218599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775c042-0264-4c82-ac3a-4b245578b23c admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:55.228391 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1004/2025] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:55.232649 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47223c5b-3475-4f2c-8f32-3f3d0136f703 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:55.243291 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47223c5b-3475-4f2c-8f32-3f3d0136f703 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:55.248390 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11e500-6536-4902-99f9-63f0308d4b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:55.248995 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11e500-6536-4902-99f9-63f0308d4b3a admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:55.249273 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11e500-6536-4902-99f9-63f0308d4b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:55.249641 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11e500-6536-4902-99f9-63f0308d4b3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:55.250383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d11e500-6536-4902-99f9-63f0308d4b3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:55.251953 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d11e500-6536-4902-99f9-63f0308d4b3a admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:55.262438 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1022/2026] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:55.266489 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775c042-0264-4c82-ac3a-4b245578b23c admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:55.276769 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775c042-0264-4c82-ac3a-4b245578b23c admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:55.281292 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35484b-eaad-4846-8da0-eaec519f1c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:55.281926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35484b-eaad-4846-8da0-eaec519f1c72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:55.282235 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35484b-eaad-4846-8da0-eaec519f1c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:55.282506 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35484b-eaad-4846-8da0-eaec519f1c72 admin admin] PATH_INFO: `/v3/projects/8a678113f36746afb64e896ad63891bc/users/cf985ed87a4c4829869e6910fa334fe9/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:55.297469 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b35484b-eaad-4846-8da0-eaec519f1c72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a678113f36746afb64e896ad63891bc, user_id=cf985ed87a4c4829869e6910fa334fe9, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:55.300486 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b35484b-eaad-4846-8da0-eaec519f1c72 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:55.313702 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1005/2027] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:55 2022] PUT /identity/v3/projects/8a678113f36746afb64e896ad63891bc/users/cf985ed87a4c4829869e6910fa334fe9/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:55.324856 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dbc75-2101-44c4-bfc3-c3f85662cfe1 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:55.325173 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dbc75-2101-44c4-bfc3-c3f85662cfe1 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:55.325446 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dbc75-2101-44c4-bfc3-c3f85662cfe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:55.357398 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-874dbc75-2101-44c4-bfc3-c3f85662cfe1 None None] MFA Rules not processed for user `cf985ed87a4c4829869e6910fa334fe9`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:55.358645 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-874dbc75-2101-44c4-bfc3-c3f85662cfe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:55.362551 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:55.425346 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1023/2028] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:55 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:55.430104 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35484b-eaad-4846-8da0-eaec519f1c72 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:55.492720 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35484b-eaad-4846-8da0-eaec519f1c72 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:55.497378 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d43cfa-7b42-4467-a4ba-b5bc94b7a480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:55.498163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d43cfa-7b42-4467-a4ba-b5bc94b7a480 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:55.498293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d43cfa-7b42-4467-a4ba-b5bc94b7a480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:55.498548 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d43cfa-7b42-4467-a4ba-b5bc94b7a480 admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:55.499222 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d43cfa-7b42-4467-a4ba-b5bc94b7a480 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:55.501143 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d43cfa-7b42-4467-a4ba-b5bc94b7a480 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:55.528810 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:55.536811 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1006/2029] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:33:55 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:55.540991 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874dbc75-2101-44c4-bfc3-c3f85662cfe1 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:55.551549 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874dbc75-2101-44c4-bfc3-c3f85662cfe1 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:55.617188 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b05d1f-7a08-4bb6-8c8a-a30d502fdc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:55.618111 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b05d1f-7a08-4bb6-8c8a-a30d502fdc1e admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:55.618907 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b05d1f-7a08-4bb6-8c8a-a30d502fdc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:55.618907 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b05d1f-7a08-4bb6-8c8a-a30d502fdc1e admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:55.619660 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b05d1f-7a08-4bb6-8c8a-a30d502fdc1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:55.622185 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b05d1f-7a08-4bb6-8c8a-a30d502fdc1e admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:55.657403 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1024/2030] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:33:55 2022] POST /identity/v3/users => generated 300 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:33:55.658762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d43cfa-7b42-4467-a4ba-b5bc94b7a480 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:55.672618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d43cfa-7b42-4467-a4ba-b5bc94b7a480 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:55.677519 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1f397-4e3b-4ecd-b0cb-fd6289dde24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:55.678176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1f397-4e3b-4ecd-b0cb-fd6289dde24d admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:55.678449 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1f397-4e3b-4ecd-b0cb-fd6289dde24d admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:55.678714 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1f397-4e3b-4ecd-b0cb-fd6289dde24d admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:55.679386 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c1f397-4e3b-4ecd-b0cb-fd6289dde24d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:55.681043 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c1f397-4e3b-4ecd-b0cb-fd6289dde24d admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:55.696102 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1007/2031] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:55.697365 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b05d1f-7a08-4bb6-8c8a-a30d502fdc1e admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:55.708565 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b05d1f-7a08-4bb6-8c8a-a30d502fdc1e admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:55.714533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d4816-0050-48b0-b636-cf7676c10971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:55.715249 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d4816-0050-48b0-b636-cf7676c10971 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:55.715531 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d4816-0050-48b0-b636-cf7676c10971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:55.715810 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d4816-0050-48b0-b636-cf7676c10971 admin admin] PATH_INFO: `/v3/projects/561c982b454e46d7b542f43045564f9c/users/c0dd528b7e2c4a7fb28d7933e9608734/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:55.735291 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d4816-0050-48b0-b636-cf7676c10971 admin admin] RBAC: Authorizing `identity:create_grant(project_id=561c982b454e46d7b542f43045564f9c, user_id=c0dd528b7e2c4a7fb28d7933e9608734, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:55.737987 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d4816-0050-48b0-b636-cf7676c10971 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:55.752623 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1025/2032] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:55 2022] PUT /identity/v3/projects/561c982b454e46d7b542f43045564f9c/users/c0dd528b7e2c4a7fb28d7933e9608734/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:55.756553 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1f397-4e3b-4ecd-b0cb-fd6289dde24d admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:55.767677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1f397-4e3b-4ecd-b0cb-fd6289dde24d admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:55.827062 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc74be1-bf30-4b41-962e-ef069d1d6af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:55.827670 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc74be1-bf30-4b41-962e-ef069d1d6af7 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:55.827952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc74be1-bf30-4b41-962e-ef069d1d6af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:55.828226 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc74be1-bf30-4b41-962e-ef069d1d6af7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:55.828867 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc74be1-bf30-4b41-962e-ef069d1d6af7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:55.830501 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc74be1-bf30-4b41-962e-ef069d1d6af7 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:55.840512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1008/2033] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:55.845631 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d4816-0050-48b0-b636-cf7676c10971 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:55.856392 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d4816-0050-48b0-b636-cf7676c10971 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:55.862423 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a13b4f9-7733-40fd-af36-215082b7e88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:55.863286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13b4f9-7733-40fd-af36-215082b7e88e admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:55.863562 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13b4f9-7733-40fd-af36-215082b7e88e admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:55.863897 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13b4f9-7733-40fd-af36-215082b7e88e admin admin] PATH_INFO: `/v3/projects/561c982b454e46d7b542f43045564f9c/users/c0dd528b7e2c4a7fb28d7933e9608734/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:55.866494 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a13b4f9-7733-40fd-af36-215082b7e88e admin admin] RBAC: Authorizing `identity:create_grant(project_id=561c982b454e46d7b542f43045564f9c, user_id=c0dd528b7e2c4a7fb28d7933e9608734, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:55.869135 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a13b4f9-7733-40fd-af36-215082b7e88e admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:55.879936 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1026/2034] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:55 2022] PUT /identity/v3/projects/561c982b454e46d7b542f43045564f9c/users/c0dd528b7e2c4a7fb28d7933e9608734/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:55.890743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bff9ea-ce00-4a0f-8990-cfadaa369bb7 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:55.891032 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bff9ea-ce00-4a0f-8990-cfadaa369bb7 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:55.891332 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bff9ea-ce00-4a0f-8990-cfadaa369bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:55.919041 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-01bff9ea-ce00-4a0f-8990-cfadaa369bb7 None None] MFA Rules not processed for user `c0dd528b7e2c4a7fb28d7933e9608734`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:55.920253 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-01bff9ea-ce00-4a0f-8990-cfadaa369bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:55.990906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1009/2035] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:55 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:56.001965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a13b4f9-7733-40fd-af36-215082b7e88e admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:56.003912 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-0a13b4f9-7733-40fd-af36-215082b7e88e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:56.064481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a13b4f9-7733-40fd-af36-215082b7e88e admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:56.068401 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aee816-d325-4d76-a7ff-a10dc2d54df3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:56.069196 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aee816-d325-4d76-a7ff-a10dc2d54df3 service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:56.069394 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aee816-d325-4d76-a7ff-a10dc2d54df3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:56.069663 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aee816-d325-4d76-a7ff-a10dc2d54df3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:56.081662 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aee816-d325-4d76-a7ff-a10dc2d54df3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:56.083170 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aee816-d325-4d76-a7ff-a10dc2d54df3 service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:56.090532 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:56.094236 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1027/2036] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:56 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:33:56.919663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bff9ea-ce00-4a0f-8990-cfadaa369bb7 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:56.984060 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bff9ea-ce00-4a0f-8990-cfadaa369bb7 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:56.989481 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5893e45-d637-4576-8559-ad7a80e1774a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:56.990157 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5893e45-d637-4576-8559-ad7a80e1774a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:56.990575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5893e45-d637-4576-8559-ad7a80e1774a admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:56.991047 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5893e45-d637-4576-8559-ad7a80e1774a admin admin] PATH_INFO: `/v3/users/cf985ed87a4c4829869e6910fa334fe9` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:56.995529 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5893e45-d637-4576-8559-ad7a80e1774a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf985ed87a4c4829869e6910fa334fe9)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:56.997525 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5893e45-d637-4576-8559-ad7a80e1774a admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:57.012877 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:57.047445 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e5893e45-d637-4576-8559-ad7a80e1774a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf985ed87a4c4829869e6910fa334fe9'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:57.048016 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e5893e45-d637-4576-8559-ad7a80e1774a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf985ed87a4c4829869e6910fa334fe9'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:57.048619 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e5893e45-d637-4576-8559-ad7a80e1774a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf985ed87a4c4829869e6910fa334fe9'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:57.057422 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e5893e45-d637-4576-8559-ad7a80e1774a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf985ed87a4c4829869e6910fa334fe9'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:57.068150 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e5893e45-d637-4576-8559-ad7a80e1774a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf985ed87a4c4829869e6910fa334fe9'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:57.083385 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1010/2037] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:33:56 2022] DELETE /identity/v3/users/cf985ed87a4c4829869e6910fa334fe9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:57.088141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12412f3-f73b-4c99-9f9f-4b021d23549a None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:57.089263 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12412f3-f73b-4c99-9f9f-4b021d23549a None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:57.089636 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12412f3-f73b-4c99-9f9f-4b021d23549a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:57.109061 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-f12412f3-f73b-4c99-9f9f-4b021d23549a None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:57.110291 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-f12412f3-f73b-4c99-9f9f-4b021d23549a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:57.171674 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1028/2038] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:57.181277 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5893e45-d637-4576-8559-ad7a80e1774a admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:57.182709 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-e5893e45-d637-4576-8559-ad7a80e1774a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:57.240724 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5893e45-d637-4576-8559-ad7a80e1774a admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:57.244332 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3c573-1bbb-4fa6-8e13-0df02f575ced service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:57.244952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3c573-1bbb-4fa6-8e13-0df02f575ced service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:57.245284 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3c573-1bbb-4fa6-8e13-0df02f575ced service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:57.245557 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3c573-1bbb-4fa6-8e13-0df02f575ced service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:57.254465 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b3c573-1bbb-4fa6-8e13-0df02f575ced service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:57.255886 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b3c573-1bbb-4fa6-8e13-0df02f575ced service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:57.265837 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1011/2039] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:33:57.581117 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12412f3-f73b-4c99-9f9f-4b021d23549a None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:57.582988 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-f12412f3-f73b-4c99-9f9f-4b021d23549a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:57.595808 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12412f3-f73b-4c99-9f9f-4b021d23549a None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:57.603014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a842d261-2a18-44d6-93a6-3925884fc72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:57.603711 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a842d261-2a18-44d6-93a6-3925884fc72c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:57.603996 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a842d261-2a18-44d6-93a6-3925884fc72c admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:57.604272 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a842d261-2a18-44d6-93a6-3925884fc72c admin admin] PATH_INFO: `/v3/projects/8a678113f36746afb64e896ad63891bc` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:57.605475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a842d261-2a18-44d6-93a6-3925884fc72c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a678113f36746afb64e896ad63891bc)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:57.608279 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a842d261-2a18-44d6-93a6-3925884fc72c admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:57.657603 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-a842d261-2a18-44d6-93a6-3925884fc72c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a678113f36746afb64e896ad63891bc', 'request_id': 'req-a842d261-2a18-44d6-93a6-3925884fc72c'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:57.663849 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-a842d261-2a18-44d6-93a6-3925884fc72c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a678113f36746afb64e896ad63891bc', 'request_id': 'req-a842d261-2a18-44d6-93a6-3925884fc72c'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:57.672142 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-a842d261-2a18-44d6-93a6-3925884fc72c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a678113f36746afb64e896ad63891bc', 'request_id': 'req-a842d261-2a18-44d6-93a6-3925884fc72c'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:57.683663 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-a842d261-2a18-44d6-93a6-3925884fc72c admin admin] The token cache is being invalidate because project 8a678113f36746afb64e896ad63891bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:33:57.683953 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-a842d261-2a18-44d6-93a6-3925884fc72c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:57.684584 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-a842d261-2a18-44d6-93a6-3925884fc72c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:57.686084 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1029/2040] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:33:57 2022] DELETE /identity/v3/projects/8a678113f36746afb64e896ad63891bc => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:57.690384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3c573-1bbb-4fa6-8e13-0df02f575ced service neutron] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:57.699991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3c573-1bbb-4fa6-8e13-0df02f575ced service neutron] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:57.761675 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f13c4-d635-49e6-9ab4-e892d35d6908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:57.762340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f13c4-d635-49e6-9ab4-e892d35d6908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:57.762618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f13c4-d635-49e6-9ab4-e892d35d6908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:57.762889 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f13c4-d635-49e6-9ab4-e892d35d6908 admin admin] PATH_INFO: `/v3/users/c0dd528b7e2c4a7fb28d7933e9608734` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:57.763989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f13c4-d635-49e6-9ab4-e892d35d6908 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0dd528b7e2c4a7fb28d7933e9608734)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:57.765903 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f13c4-d635-49e6-9ab4-e892d35d6908 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:57.805171 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:57.815517 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-ac1f13c4-d635-49e6-9ab4-e892d35d6908 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0dd528b7e2c4a7fb28d7933e9608734'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:57.816645 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-ac1f13c4-d635-49e6-9ab4-e892d35d6908 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0dd528b7e2c4a7fb28d7933e9608734'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:57.817262 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-ac1f13c4-d635-49e6-9ab4-e892d35d6908 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0dd528b7e2c4a7fb28d7933e9608734'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:57.824268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-ac1f13c4-d635-49e6-9ab4-e892d35d6908 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0dd528b7e2c4a7fb28d7933e9608734'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:57.835886 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-ac1f13c4-d635-49e6-9ab4-e892d35d6908 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0dd528b7e2c4a7fb28d7933e9608734'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:57.848190 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1012/2041] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:33:57 2022] DELETE /identity/v3/users/c0dd528b7e2c4a7fb28d7933e9608734 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:58.128291 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a842d261-2a18-44d6-93a6-3925884fc72c admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:58.129820 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-a842d261-2a18-44d6-93a6-3925884fc72c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:58.143938 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a842d261-2a18-44d6-93a6-3925884fc72c admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:58.203229 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f520c438-872c-49ea-96e9-16b3ace86193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:58.203863 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520c438-872c-49ea-96e9-16b3ace86193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:58.204152 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520c438-872c-49ea-96e9-16b3ace86193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:58.204413 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520c438-872c-49ea-96e9-16b3ace86193 admin admin] PATH_INFO: `/v3/projects/561c982b454e46d7b542f43045564f9c` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:58.205537 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f520c438-872c-49ea-96e9-16b3ace86193 admin admin] RBAC: Authorizing `identity:delete_project(project_id=561c982b454e46d7b542f43045564f9c)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:58.207451 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f520c438-872c-49ea-96e9-16b3ace86193 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:58.240253 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:58.258476 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f520c438-872c-49ea-96e9-16b3ace86193 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '561c982b454e46d7b542f43045564f9c', 'request_id': 'req-f520c438-872c-49ea-96e9-16b3ace86193'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:58.264796 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f520c438-872c-49ea-96e9-16b3ace86193 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '561c982b454e46d7b542f43045564f9c', 'request_id': 'req-f520c438-872c-49ea-96e9-16b3ace86193'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:58.272357 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f520c438-872c-49ea-96e9-16b3ace86193 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '561c982b454e46d7b542f43045564f9c', 'request_id': 'req-f520c438-872c-49ea-96e9-16b3ace86193'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:58.283570 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f520c438-872c-49ea-96e9-16b3ace86193 admin admin] The token cache is being invalidate because project 561c982b454e46d7b542f43045564f9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:33:58.283861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f520c438-872c-49ea-96e9-16b3ace86193 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:58.284495 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f520c438-872c-49ea-96e9-16b3ace86193 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:58.286272 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1030/2042] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:33:58 2022] DELETE /identity/v3/projects/561c982b454e46d7b542f43045564f9c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:33:58.291294 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0279c-0c60-48fb-857d-0e2a4d7b9027 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:58.292348 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0279c-0c60-48fb-857d-0e2a4d7b9027 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:58.292544 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0279c-0c60-48fb-857d-0e2a4d7b9027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:58.312556 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-2ba0279c-0c60-48fb-857d-0e2a4d7b9027 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:58.314228 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-2ba0279c-0c60-48fb-857d-0e2a4d7b9027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:58.374428 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1013/2043] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:58.381845 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb90c7c-3879-47cd-a029-5f14c7c45488 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:58.382168 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb90c7c-3879-47cd-a029-5f14c7c45488 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:58.382447 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb90c7c-3879-47cd-a029-5f14c7c45488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:58.403723 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-4eb90c7c-3879-47cd-a029-5f14c7c45488 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:58.404919 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-4eb90c7c-3879-47cd-a029-5f14c7c45488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:58.412725 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1031/2044] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:58.416815 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0279c-0c60-48fb-857d-0e2a4d7b9027 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:58.426545 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0279c-0c60-48fb-857d-0e2a4d7b9027 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:58.431064 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d7c38-94cf-4024-bddf-27cbf0883555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:58.431647 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d7c38-94cf-4024-bddf-27cbf0883555 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:58.431920 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d7c38-94cf-4024-bddf-27cbf0883555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:58.432187 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d7c38-94cf-4024-bddf-27cbf0883555 admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:58.432817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d7c38-94cf-4024-bddf-27cbf0883555 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:33:58.433110 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d7c38-94cf-4024-bddf-27cbf0883555 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:58.434716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d7c38-94cf-4024-bddf-27cbf0883555 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:58.445844 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1014/2045] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:33:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:33:58.450676 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb90c7c-3879-47cd-a029-5f14c7c45488 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:58.460716 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb90c7c-3879-47cd-a029-5f14c7c45488 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:58.466008 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c5d21-a66c-4d0f-bac8-0cee0506c815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:58.466635 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c5d21-a66c-4d0f-bac8-0cee0506c815 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:58.466912 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c5d21-a66c-4d0f-bac8-0cee0506c815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:58.467186 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c5d21-a66c-4d0f-bac8-0cee0506c815 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:58.467817 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c5d21-a66c-4d0f-bac8-0cee0506c815 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:58.469703 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c5d21-a66c-4d0f-bac8-0cee0506c815 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:58.507302 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1032/2046] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:33:58 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:58.511901 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d7c38-94cf-4024-bddf-27cbf0883555 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:58.521880 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d7c38-94cf-4024-bddf-27cbf0883555 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:58.577760 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a114a5-d2dc-4c93-8a9a-77cee16ec574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:58.578394 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a114a5-d2dc-4c93-8a9a-77cee16ec574 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:58.578671 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a114a5-d2dc-4c93-8a9a-77cee16ec574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:58.579023 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a114a5-d2dc-4c93-8a9a-77cee16ec574 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:58.579693 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a114a5-d2dc-4c93-8a9a-77cee16ec574 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:58.581611 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a114a5-d2dc-4c93-8a9a-77cee16ec574 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:58.603930 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1015/2047] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:33:58 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:58.607949 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c5d21-a66c-4d0f-bac8-0cee0506c815 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:58.618180 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c5d21-a66c-4d0f-bac8-0cee0506c815 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:58.622756 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e4e63-5b4f-4c87-8e08-9ebb0ee1df81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:58.623370 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e4e63-5b4f-4c87-8e08-9ebb0ee1df81 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:58.623698 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e4e63-5b4f-4c87-8e08-9ebb0ee1df81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:58.623966 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e4e63-5b4f-4c87-8e08-9ebb0ee1df81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:58.624699 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e4e63-5b4f-4c87-8e08-9ebb0ee1df81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:58.626243 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e4e63-5b4f-4c87-8e08-9ebb0ee1df81 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:58.637399 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1033/2048] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:58.641311 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a114a5-d2dc-4c93-8a9a-77cee16ec574 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:58.652117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a114a5-d2dc-4c93-8a9a-77cee16ec574 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:58.656923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c9d0b-5a27-4827-bdff-9e66b937decd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:58.657521 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c9d0b-5a27-4827-bdff-9e66b937decd admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:58.657796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c9d0b-5a27-4827-bdff-9e66b937decd admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:58.658105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c9d0b-5a27-4827-bdff-9e66b937decd admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:58.659021 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c9d0b-5a27-4827-bdff-9e66b937decd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:58.660441 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c9d0b-5a27-4827-bdff-9e66b937decd admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:58.671760 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1016/2049] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:33:58.675887 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e4e63-5b4f-4c87-8e08-9ebb0ee1df81 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:58.686137 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e4e63-5b4f-4c87-8e08-9ebb0ee1df81 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:58.690797 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce71b00a-2cdb-4160-953f-2f1682e45a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:58.691451 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71b00a-2cdb-4160-953f-2f1682e45a9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:58.691723 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71b00a-2cdb-4160-953f-2f1682e45a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:58.692001 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71b00a-2cdb-4160-953f-2f1682e45a9f admin admin] PATH_INFO: `/v3/projects/04c4970e7f1543d7a27968818da61102/users/eafacb34b51e49e2a18d3f8fb2fc1cdf/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:58.707498 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce71b00a-2cdb-4160-953f-2f1682e45a9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=04c4970e7f1543d7a27968818da61102, user_id=eafacb34b51e49e2a18d3f8fb2fc1cdf, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:58.710316 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce71b00a-2cdb-4160-953f-2f1682e45a9f admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:58.728377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1034/2050] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:33:58 2022] PUT /identity/v3/projects/04c4970e7f1543d7a27968818da61102/users/eafacb34b51e49e2a18d3f8fb2fc1cdf/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:58.739477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11844e38-3295-4bc8-902f-6374127c2154 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:58.739820 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11844e38-3295-4bc8-902f-6374127c2154 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:58.740185 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11844e38-3295-4bc8-902f-6374127c2154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:58.781902 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-11844e38-3295-4bc8-902f-6374127c2154 None None] MFA Rules not processed for user `eafacb34b51e49e2a18d3f8fb2fc1cdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:58.783316 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-11844e38-3295-4bc8-902f-6374127c2154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:58.789012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:58.866410 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1017/2051] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:58.878123 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce71b00a-2cdb-4160-953f-2f1682e45a9f admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:58.879749 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-ce71b00a-2cdb-4160-953f-2f1682e45a9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:58.936659 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce71b00a-2cdb-4160-953f-2f1682e45a9f admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:58.940292 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b98b7f5-60e4-416f-8579-f2a7e799b46f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:58.940897 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98b7f5-60e4-416f-8579-f2a7e799b46f service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:58.941173 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98b7f5-60e4-416f-8579-f2a7e799b46f service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:58.941447 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98b7f5-60e4-416f-8579-f2a7e799b46f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:58.950324 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b98b7f5-60e4-416f-8579-f2a7e799b46f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:58.951735 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b98b7f5-60e4-416f-8579-f2a7e799b46f service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:58.961269 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1035/2052] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:58 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:33:58.985173 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11844e38-3295-4bc8-902f-6374127c2154 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:59.047103 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11844e38-3295-4bc8-902f-6374127c2154 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:59.051710 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd54f6f-960b-4b4a-bd3c-5c531c0219dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:59.052330 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd54f6f-960b-4b4a-bd3c-5c531c0219dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:59.052608 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd54f6f-960b-4b4a-bd3c-5c531c0219dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:59.052881 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd54f6f-960b-4b4a-bd3c-5c531c0219dd admin admin] PATH_INFO: `/v3/users/eafacb34b51e49e2a18d3f8fb2fc1cdf` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:59.053994 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd54f6f-960b-4b4a-bd3c-5c531c0219dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=eafacb34b51e49e2a18d3f8fb2fc1cdf)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:59.055913 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd54f6f-960b-4b4a-bd3c-5c531c0219dd admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:59.101636 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-4cd54f6f-960b-4b4a-bd3c-5c531c0219dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eafacb34b51e49e2a18d3f8fb2fc1cdf'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:59.102316 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-4cd54f6f-960b-4b4a-bd3c-5c531c0219dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eafacb34b51e49e2a18d3f8fb2fc1cdf'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:59.102931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-4cd54f6f-960b-4b4a-bd3c-5c531c0219dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eafacb34b51e49e2a18d3f8fb2fc1cdf'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:59.109775 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-4cd54f6f-960b-4b4a-bd3c-5c531c0219dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eafacb34b51e49e2a18d3f8fb2fc1cdf'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:59.122071 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-4cd54f6f-960b-4b4a-bd3c-5c531c0219dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eafacb34b51e49e2a18d3f8fb2fc1cdf'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:59.134065 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1018/2053] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:33:58 2022] DELETE /identity/v3/users/eafacb34b51e49e2a18d3f8fb2fc1cdf => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:59.139175 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a4a88-bb18-4305-92a2-bf363f8516e5 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:59.139628 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a4a88-bb18-4305-92a2-bf363f8516e5 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:59.139718 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a4a88-bb18-4305-92a2-bf363f8516e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:59.158976 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-7b8a4a88-bb18-4305-92a2-bf363f8516e5 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:59.160230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-7b8a4a88-bb18-4305-92a2-bf363f8516e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:59.207038 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:59.220603 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1036/2054] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:59.230914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd54f6f-960b-4b4a-bd3c-5c531c0219dd admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:59.232301 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-4cd54f6f-960b-4b4a-bd3c-5c531c0219dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:59.263587 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:33:59.291430 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd54f6f-960b-4b4a-bd3c-5c531c0219dd admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:59.295630 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc046c0-210f-4633-9061-7998c601274d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:59.296228 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc046c0-210f-4633-9061-7998c601274d service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:59.296506 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc046c0-210f-4633-9061-7998c601274d service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:59.296778 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc046c0-210f-4633-9061-7998c601274d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:59.305635 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc046c0-210f-4633-9061-7998c601274d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:59.307089 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc046c0-210f-4633-9061-7998c601274d service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:59.317086 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1019/2055] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:33:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:33:59.591476 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a4a88-bb18-4305-92a2-bf363f8516e5 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:59.592982 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-7b8a4a88-bb18-4305-92a2-bf363f8516e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:59.605963 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a4a88-bb18-4305-92a2-bf363f8516e5 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:59.611412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af961525-b51c-415d-9907-97fd89b8e088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:59.611636 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af961525-b51c-415d-9907-97fd89b8e088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:59.611904 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af961525-b51c-415d-9907-97fd89b8e088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:59.612176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af961525-b51c-415d-9907-97fd89b8e088 admin admin] PATH_INFO: `/v3/projects/04c4970e7f1543d7a27968818da61102` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:59.613303 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af961525-b51c-415d-9907-97fd89b8e088 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04c4970e7f1543d7a27968818da61102)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:59.615327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af961525-b51c-415d-9907-97fd89b8e088 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:59.670618 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-af961525-b51c-415d-9907-97fd89b8e088 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04c4970e7f1543d7a27968818da61102', 'request_id': 'req-af961525-b51c-415d-9907-97fd89b8e088'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:59.676545 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-af961525-b51c-415d-9907-97fd89b8e088 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04c4970e7f1543d7a27968818da61102', 'request_id': 'req-af961525-b51c-415d-9907-97fd89b8e088'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:59.683176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-af961525-b51c-415d-9907-97fd89b8e088 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04c4970e7f1543d7a27968818da61102', 'request_id': 'req-af961525-b51c-415d-9907-97fd89b8e088'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:59.693395 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-af961525-b51c-415d-9907-97fd89b8e088 admin admin] The token cache is being invalidate because project 04c4970e7f1543d7a27968818da61102 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:33:59.693679 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-af961525-b51c-415d-9907-97fd89b8e088 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:59.694306 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-af961525-b51c-415d-9907-97fd89b8e088 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:33:59.695769 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1037/2056] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:33:59 2022] DELETE /identity/v3/projects/04c4970e7f1543d7a27968818da61102 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:33:59.700770 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed4f35-3dc0-4493-b147-0fa17915397d None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:59.701054 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed4f35-3dc0-4493-b147-0fa17915397d None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:59.701326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed4f35-3dc0-4493-b147-0fa17915397d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:59.721969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-a1ed4f35-3dc0-4493-b147-0fa17915397d None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:59.723265 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-a1ed4f35-3dc0-4493-b147-0fa17915397d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:59.786260 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1020/2057] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:59.794839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e91e5-6c01-4dec-a868-5565d047cab9 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:59.795204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e91e5-6c01-4dec-a868-5565d047cab9 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:59.795547 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e91e5-6c01-4dec-a868-5565d047cab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:59.816762 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-e97e91e5-6c01-4dec-a868-5565d047cab9 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:33:59.819017 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-e97e91e5-6c01-4dec-a868-5565d047cab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:33:59.827149 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1038/2058] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:33:59.831325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed4f35-3dc0-4493-b147-0fa17915397d None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:59.841641 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed4f35-3dc0-4493-b147-0fa17915397d None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:59.846488 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c68264-5c9f-478a-8885-e0767396c7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:59.846933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c68264-5c9f-478a-8885-e0767396c7de admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:59.847212 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c68264-5c9f-478a-8885-e0767396c7de admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:59.847483 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c68264-5c9f-478a-8885-e0767396c7de admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:59.848124 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c68264-5c9f-478a-8885-e0767396c7de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:33:59.848426 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c68264-5c9f-478a-8885-e0767396c7de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:59.850370 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c68264-5c9f-478a-8885-e0767396c7de admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:59.862858 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1021/2059] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:33:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:33:59.867280 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e91e5-6c01-4dec-a868-5565d047cab9 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:59.876772 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e91e5-6c01-4dec-a868-5565d047cab9 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:59.881533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d6d44-8969-499d-b4b3-c669e617c715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:59.882252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d6d44-8969-499d-b4b3-c669e617c715 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:59.882534 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d6d44-8969-499d-b4b3-c669e617c715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:59.882805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d6d44-8969-499d-b4b3-c669e617c715 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:59.883473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d6d44-8969-499d-b4b3-c669e617c715 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:33:59.885361 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d6d44-8969-499d-b4b3-c669e617c715 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:33:59.915369 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1039/2060] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:33:59 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:33:59.919558 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c68264-5c9f-478a-8885-e0767396c7de admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:33:59.930152 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c68264-5c9f-478a-8885-e0767396c7de admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:33:59.996641 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e3ef8-4708-4731-865d-ea42539e0db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:33:59.997303 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e3ef8-4708-4731-865d-ea42539e0db8 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:33:59.997533 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e3ef8-4708-4731-865d-ea42539e0db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:33:59.997811 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e3ef8-4708-4731-865d-ea42539e0db8 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:33:59.998498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733e3ef8-4708-4731-865d-ea42539e0db8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:00.000408 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733e3ef8-4708-4731-865d-ea42539e0db8 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:00.024243 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1022/2061] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:33:59 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:34:00.028972 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d6d44-8969-499d-b4b3-c669e617c715 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:00.039968 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d6d44-8969-499d-b4b3-c669e617c715 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:00.044673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da8533-7d14-4b35-ad5b-de835613feee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:00.045267 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da8533-7d14-4b35-ad5b-de835613feee admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:00.045547 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da8533-7d14-4b35-ad5b-de835613feee admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:00.045828 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da8533-7d14-4b35-ad5b-de835613feee admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:00.046497 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da8533-7d14-4b35-ad5b-de835613feee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:00.048075 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da8533-7d14-4b35-ad5b-de835613feee admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:00.061065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1040/2062] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:34:00.062283 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e3ef8-4708-4731-865d-ea42539e0db8 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:00.072408 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e3ef8-4708-4731-865d-ea42539e0db8 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:00.077513 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e233b5-bccd-4670-a6d3-e0c471974a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:00.078122 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e233b5-bccd-4670-a6d3-e0c471974a82 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:00.078404 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e233b5-bccd-4670-a6d3-e0c471974a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:00.078672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e233b5-bccd-4670-a6d3-e0c471974a82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:00.079325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e233b5-bccd-4670-a6d3-e0c471974a82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:00.080897 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e233b5-bccd-4670-a6d3-e0c471974a82 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:00.091940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1023/2063] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:34:00.095969 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da8533-7d14-4b35-ad5b-de835613feee admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:00.106615 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da8533-7d14-4b35-ad5b-de835613feee admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:00.111342 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63872c9-b54c-4c2f-b929-5345d83c2d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:00.111977 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63872c9-b54c-4c2f-b929-5345d83c2d66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:00.112255 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63872c9-b54c-4c2f-b929-5345d83c2d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:00.112528 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63872c9-b54c-4c2f-b929-5345d83c2d66 admin admin] PATH_INFO: `/v3/projects/42e1027669fb454899c6a54c410d73ed/users/6eef257db6c6436ea91fcf267632e67a/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:00.127062 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63872c9-b54c-4c2f-b929-5345d83c2d66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42e1027669fb454899c6a54c410d73ed, user_id=6eef257db6c6436ea91fcf267632e67a, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:00.129655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63872c9-b54c-4c2f-b929-5345d83c2d66 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:00.142858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1041/2064] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:34:00 2022] PUT /identity/v3/projects/42e1027669fb454899c6a54c410d73ed/users/6eef257db6c6436ea91fcf267632e67a/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:00.155305 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbc623-58d1-4bff-b95a-49b6dfaf673a None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:00.155594 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbc623-58d1-4bff-b95a-49b6dfaf673a None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:00.155871 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbc623-58d1-4bff-b95a-49b6dfaf673a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:00.184130 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-b6cbc623-58d1-4bff-b95a-49b6dfaf673a None None] MFA Rules not processed for user `6eef257db6c6436ea91fcf267632e67a`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:00.185305 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-b6cbc623-58d1-4bff-b95a-49b6dfaf673a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:00.210626 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:00.253951 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1024/2065] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:34:00.258796 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63872c9-b54c-4c2f-b929-5345d83c2d66 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:00.326651 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63872c9-b54c-4c2f-b929-5345d83c2d66 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:00.331359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a1dae-5ac0-4102-91ad-759afe69af7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:00.331965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a1dae-5ac0-4102-91ad-759afe69af7b admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:00.332248 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a1dae-5ac0-4102-91ad-759afe69af7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:00.332523 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a1dae-5ac0-4102-91ad-759afe69af7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:00.333156 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a1dae-5ac0-4102-91ad-759afe69af7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:00.335070 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a1dae-5ac0-4102-91ad-759afe69af7b admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:00.355059 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:00.365312 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1042/2066] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:34:00 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:34:00.366459 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cbc623-58d1-4bff-b95a-49b6dfaf673a None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:00.377667 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cbc623-58d1-4bff-b95a-49b6dfaf673a None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:00.434101 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2313383-4715-4877-8105-e095b8da5b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:00.434762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2313383-4715-4877-8105-e095b8da5b87 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:00.434983 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2313383-4715-4877-8105-e095b8da5b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:00.435256 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2313383-4715-4877-8105-e095b8da5b87 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:00.435879 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2313383-4715-4877-8105-e095b8da5b87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:00.437820 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2313383-4715-4877-8105-e095b8da5b87 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:00.458986 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1025/2067] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:34:00 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:34:00.462933 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a1dae-5ac0-4102-91ad-759afe69af7b admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:00.473198 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a1dae-5ac0-4102-91ad-759afe69af7b admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:00.478178 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f74144-b86d-4e16-aefc-a28700287dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:00.478772 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f74144-b86d-4e16-aefc-a28700287dcb admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:00.479055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f74144-b86d-4e16-aefc-a28700287dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:00.480905 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f74144-b86d-4e16-aefc-a28700287dcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:00.480905 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f74144-b86d-4e16-aefc-a28700287dcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:00.481544 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f74144-b86d-4e16-aefc-a28700287dcb admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:00.496171 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1043/2068] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:34:00.501336 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2313383-4715-4877-8105-e095b8da5b87 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:00.512511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2313383-4715-4877-8105-e095b8da5b87 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:00.517269 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82471758-6707-4297-a113-848e5b134c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:00.518076 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82471758-6707-4297-a113-848e5b134c77 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:00.518504 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82471758-6707-4297-a113-848e5b134c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:00.519085 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82471758-6707-4297-a113-848e5b134c77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:00.519575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82471758-6707-4297-a113-848e5b134c77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:00.521585 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82471758-6707-4297-a113-848e5b134c77 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:00.532140 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1026/2069] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:34:00.536236 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f74144-b86d-4e16-aefc-a28700287dcb admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:00.546389 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f74144-b86d-4e16-aefc-a28700287dcb admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:00.551187 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27045fc-6e46-49f8-a0ac-283491c8ff81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:00.551835 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27045fc-6e46-49f8-a0ac-283491c8ff81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:00.552116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27045fc-6e46-49f8-a0ac-283491c8ff81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:00.552399 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27045fc-6e46-49f8-a0ac-283491c8ff81 admin admin] PATH_INFO: `/v3/projects/69acb26f858847b58671f6ab36ee68e4/users/6bbe23a7b451494e954fd47eab70b644/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:00.568885 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27045fc-6e46-49f8-a0ac-283491c8ff81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69acb26f858847b58671f6ab36ee68e4, user_id=6bbe23a7b451494e954fd47eab70b644, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:00.571588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27045fc-6e46-49f8-a0ac-283491c8ff81 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:00.586682 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1044/2070] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:34:00 2022] PUT /identity/v3/projects/69acb26f858847b58671f6ab36ee68e4/users/6bbe23a7b451494e954fd47eab70b644/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:00.596593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f22b4a-bcea-4bde-9e7a-30227e390b53 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:00.596876 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f22b4a-bcea-4bde-9e7a-30227e390b53 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:00.597156 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f22b4a-bcea-4bde-9e7a-30227e390b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:00.625793 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-50f22b4a-bcea-4bde-9e7a-30227e390b53 None None] MFA Rules not processed for user `6bbe23a7b451494e954fd47eab70b644`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:00.627053 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-50f22b4a-bcea-4bde-9e7a-30227e390b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:00.699394 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1027/2071] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:34:00.705092 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27045fc-6e46-49f8-a0ac-283491c8ff81 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:00.770374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27045fc-6e46-49f8-a0ac-283491c8ff81 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:00.775148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be124d9-4f3d-4dcf-b313-4727bc4c188e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:00.775749 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be124d9-4f3d-4dcf-b313-4727bc4c188e admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:00.776027 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be124d9-4f3d-4dcf-b313-4727bc4c188e admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:00.776300 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be124d9-4f3d-4dcf-b313-4727bc4c188e admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:00.776938 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be124d9-4f3d-4dcf-b313-4727bc4c188e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:00.778848 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be124d9-4f3d-4dcf-b313-4727bc4c188e admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:00.820784 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1045/2072] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:34:00 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:34:00.826092 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f22b4a-bcea-4bde-9e7a-30227e390b53 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:00.845137 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f22b4a-bcea-4bde-9e7a-30227e390b53 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:00.882527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:00.910860 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1433b8-49fb-4754-a6d5-3de67a6086b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:00.911501 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1433b8-49fb-4754-a6d5-3de67a6086b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:00.911783 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1433b8-49fb-4754-a6d5-3de67a6086b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:00.912057 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1433b8-49fb-4754-a6d5-3de67a6086b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:00.912687 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1433b8-49fb-4754-a6d5-3de67a6086b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:00.914662 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1433b8-49fb-4754-a6d5-3de67a6086b4 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:00.938444 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1028/2073] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:34:00 2022] POST /identity/v3/users => generated 297 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:34:00.942452 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be124d9-4f3d-4dcf-b313-4727bc4c188e admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:00.953952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be124d9-4f3d-4dcf-b313-4727bc4c188e admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:00.958768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab19b37-b39b-42aa-a6ce-e0d9d830cd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:00.959364 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab19b37-b39b-42aa-a6ce-e0d9d830cd5b admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:00.959646 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab19b37-b39b-42aa-a6ce-e0d9d830cd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:00.959918 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab19b37-b39b-42aa-a6ce-e0d9d830cd5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:00.960560 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab19b37-b39b-42aa-a6ce-e0d9d830cd5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:00.962237 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab19b37-b39b-42aa-a6ce-e0d9d830cd5b admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:00.973108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1046/2074] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:34:00.977549 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1433b8-49fb-4754-a6d5-3de67a6086b4 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:00.990400 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1433b8-49fb-4754-a6d5-3de67a6086b4 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:00.996568 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5557fe5-3a92-4d4e-a66c-433497e0b877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:00.997236 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5557fe5-3a92-4d4e-a66c-433497e0b877 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:00.997520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5557fe5-3a92-4d4e-a66c-433497e0b877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:00.997797 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5557fe5-3a92-4d4e-a66c-433497e0b877 admin admin] PATH_INFO: `/v3/projects/e3a5c3a487074201b356c268b95c2a5a/users/38c21b9d4b7945d8bf6adab8eb7005b2/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:01.014263 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5557fe5-3a92-4d4e-a66c-433497e0b877 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3a5c3a487074201b356c268b95c2a5a, user_id=38c21b9d4b7945d8bf6adab8eb7005b2, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:01.017131 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5557fe5-3a92-4d4e-a66c-433497e0b877 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:01.035069 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1029/2075] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:34:00 2022] PUT /identity/v3/projects/e3a5c3a487074201b356c268b95c2a5a/users/38c21b9d4b7945d8bf6adab8eb7005b2/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:34:01.039439 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab19b37-b39b-42aa-a6ce-e0d9d830cd5b admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:01.063753 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab19b37-b39b-42aa-a6ce-e0d9d830cd5b admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:01.106144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5557fe5-3a92-4d4e-a66c-433497e0b877 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:01.108341 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-e5557fe5-3a92-4d4e-a66c-433497e0b877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:01.171118 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f03915-bd5e-4600-bd14-f644399be32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:01.172301 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f03915-bd5e-4600-bd14-f644399be32b admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:01.172724 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f03915-bd5e-4600-bd14-f644399be32b admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:01.173137 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f03915-bd5e-4600-bd14-f644399be32b admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:01.174143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f03915-bd5e-4600-bd14-f644399be32b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:01.176044 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f03915-bd5e-4600-bd14-f644399be32b admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:01.188401 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1047/2076] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:34:01.192414 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f03915-bd5e-4600-bd14-f644399be32b admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:01.205444 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f03915-bd5e-4600-bd14-f644399be32b admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:01.211527 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba64191-b394-4ef7-aa1e-0795c5f501a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:01.212693 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba64191-b394-4ef7-aa1e-0795c5f501a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:01.212693 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba64191-b394-4ef7-aa1e-0795c5f501a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:01.212693 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba64191-b394-4ef7-aa1e-0795c5f501a0 admin admin] PATH_INFO: `/v3/projects/e3a5c3a487074201b356c268b95c2a5a/users/38c21b9d4b7945d8bf6adab8eb7005b2/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:01.215354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba64191-b394-4ef7-aa1e-0795c5f501a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3a5c3a487074201b356c268b95c2a5a, user_id=38c21b9d4b7945d8bf6adab8eb7005b2, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:01.218628 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba64191-b394-4ef7-aa1e-0795c5f501a0 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:01.218747 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5557fe5-3a92-4d4e-a66c-433497e0b877 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:01.225773 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b19d6b-b863-4da3-87b5-20b8582e88a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:01.226588 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b19d6b-b863-4da3-87b5-20b8582e88a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:01.227027 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b19d6b-b863-4da3-87b5-20b8582e88a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:01.227674 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b19d6b-b863-4da3-87b5-20b8582e88a6 admin admin] PATH_INFO: `/v3/users/8ff56cf1087248eeb7d8fcdde52967d4` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:01.229012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b19d6b-b863-4da3-87b5-20b8582e88a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ff56cf1087248eeb7d8fcdde52967d4)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:01.231699 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b19d6b-b863-4da3-87b5-20b8582e88a6 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:01.235687 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1048/2077] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:34:01 2022] PUT /identity/v3/projects/e3a5c3a487074201b356c268b95c2a5a/users/38c21b9d4b7945d8bf6adab8eb7005b2/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:01.251768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917db1e9-3546-4fe2-96d2-afcb80d1b4f5 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:01.252075 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917db1e9-3546-4fe2-96d2-afcb80d1b4f5 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:01.252343 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917db1e9-3546-4fe2-96d2-afcb80d1b4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:01.288149 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-917db1e9-3546-4fe2-96d2-afcb80d1b4f5 None None] MFA Rules not processed for user `38c21b9d4b7945d8bf6adab8eb7005b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:01.289380 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-917db1e9-3546-4fe2-96d2-afcb80d1b4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:01.295853 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-76b19d6b-b863-4da3-87b5-20b8582e88a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ff56cf1087248eeb7d8fcdde52967d4'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:01.296516 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-76b19d6b-b863-4da3-87b5-20b8582e88a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ff56cf1087248eeb7d8fcdde52967d4'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:01.297197 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-76b19d6b-b863-4da3-87b5-20b8582e88a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ff56cf1087248eeb7d8fcdde52967d4'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:01.298950 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:01.306167 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-76b19d6b-b863-4da3-87b5-20b8582e88a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ff56cf1087248eeb7d8fcdde52967d4'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:01.319128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-76b19d6b-b863-4da3-87b5-20b8582e88a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ff56cf1087248eeb7d8fcdde52967d4'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:01.332028 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1030/2078] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:34:01 2022] DELETE /identity/v3/users/8ff56cf1087248eeb7d8fcdde52967d4 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:34:01.336324 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0205a0-0696-4e54-814a-be771a41df19 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:01.336615 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0205a0-0696-4e54-814a-be771a41df19 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:01.336893 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0205a0-0696-4e54-814a-be771a41df19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:01.356890 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-ba0205a0-0696-4e54-814a-be771a41df19 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:01.358121 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-ba0205a0-0696-4e54-814a-be771a41df19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:01.367009 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1049/2079] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:01 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:34:01.374982 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:01.379601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917db1e9-3546-4fe2-96d2-afcb80d1b4f5 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:01.381389 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-917db1e9-3546-4fe2-96d2-afcb80d1b4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:01.420389 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1031/2080] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:34:01.430047 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0205a0-0696-4e54-814a-be771a41df19 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:01.431416 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-ba0205a0-0696-4e54-814a-be771a41df19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:01.441190 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917db1e9-3546-4fe2-96d2-afcb80d1b4f5 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:01.488733 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0205a0-0696-4e54-814a-be771a41df19 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:01.492368 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805899f9-aaf0-4e3a-a188-8b47d980d0ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:01.493153 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805899f9-aaf0-4e3a-a188-8b47d980d0ab service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:01.493460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805899f9-aaf0-4e3a-a188-8b47d980d0ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:01.493727 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805899f9-aaf0-4e3a-a188-8b47d980d0ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:01.494283 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12afb0c3-7a38-45b6-8295-313515fd2d02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:01.494925 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afb0c3-7a38-45b6-8295-313515fd2d02 service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:01.495262 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afb0c3-7a38-45b6-8295-313515fd2d02 service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:01.495594 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afb0c3-7a38-45b6-8295-313515fd2d02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:01.497343 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-12afb0c3-7a38-45b6-8295-313515fd2d02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:01.504188 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805899f9-aaf0-4e3a-a188-8b47d980d0ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:01.505653 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805899f9-aaf0-4e3a-a188-8b47d980d0ab service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:01.516114 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1032/2081] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:34:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:34:01.559422 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12afb0c3-7a38-45b6-8295-313515fd2d02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:01.560865 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12afb0c3-7a38-45b6-8295-313515fd2d02 service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:01.569973 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1050/2082] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:34:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:34:01.871178 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805899f9-aaf0-4e3a-a188-8b47d980d0ab service neutron] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:01.877535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-805899f9-aaf0-4e3a-a188-8b47d980d0ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:01.891538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805899f9-aaf0-4e3a-a188-8b47d980d0ab service neutron] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:01.908027 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afbaa06-454c-4486-8103-fcbe3736bf7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:01.908681 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afbaa06-454c-4486-8103-fcbe3736bf7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:01.908958 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afbaa06-454c-4486-8103-fcbe3736bf7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:01.909226 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afbaa06-454c-4486-8103-fcbe3736bf7d admin admin] PATH_INFO: `/v3/projects/2510643511c4445eaced89475e51e2b6` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:01.910465 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afbaa06-454c-4486-8103-fcbe3736bf7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2510643511c4445eaced89475e51e2b6)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:01.912382 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afbaa06-454c-4486-8103-fcbe3736bf7d admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:01.987149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-1afbaa06-454c-4486-8103-fcbe3736bf7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2510643511c4445eaced89475e51e2b6', 'request_id': 'req-1afbaa06-454c-4486-8103-fcbe3736bf7d'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:01.994582 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-1afbaa06-454c-4486-8103-fcbe3736bf7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2510643511c4445eaced89475e51e2b6', 'request_id': 'req-1afbaa06-454c-4486-8103-fcbe3736bf7d'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:02.000755 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-1afbaa06-454c-4486-8103-fcbe3736bf7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2510643511c4445eaced89475e51e2b6', 'request_id': 'req-1afbaa06-454c-4486-8103-fcbe3736bf7d'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:02.012644 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-1afbaa06-454c-4486-8103-fcbe3736bf7d admin admin] The token cache is being invalidate because project 2510643511c4445eaced89475e51e2b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:34:02.012931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-1afbaa06-454c-4486-8103-fcbe3736bf7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:02.013549 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-1afbaa06-454c-4486-8103-fcbe3736bf7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:02.015060 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1033/2083] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:34:01 2022] DELETE /identity/v3/projects/2510643511c4445eaced89475e51e2b6 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:34:02.019887 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f6a5d-7d23-41a0-828b-6866e9e7b342 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:02.020174 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f6a5d-7d23-41a0-828b-6866e9e7b342 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:02.020448 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f6a5d-7d23-41a0-828b-6866e9e7b342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:02.041428 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-894f6a5d-7d23-41a0-828b-6866e9e7b342 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:02.043300 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-894f6a5d-7d23-41a0-828b-6866e9e7b342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:02.052649 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:02.103614 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1051/2084] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:34:02.111927 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a674441-6cfc-4137-95d4-9b0a45bf9c3f None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:02.112207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a674441-6cfc-4137-95d4-9b0a45bf9c3f None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:02.112475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a674441-6cfc-4137-95d4-9b0a45bf9c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:02.132285 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-0a674441-6cfc-4137-95d4-9b0a45bf9c3f None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:02.133500 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-0a674441-6cfc-4137-95d4-9b0a45bf9c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:02.141939 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1034/2085] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:34:02.145929 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f6a5d-7d23-41a0-828b-6866e9e7b342 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:02.156132 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f6a5d-7d23-41a0-828b-6866e9e7b342 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:02.160619 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d0d29-8ea0-42be-8f59-e0d541d3f0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:02.161204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d0d29-8ea0-42be-8f59-e0d541d3f0db admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:02.161480 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d0d29-8ea0-42be-8f59-e0d541d3f0db admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:02.161754 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d0d29-8ea0-42be-8f59-e0d541d3f0db admin admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:02.162434 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d0d29-8ea0-42be-8f59-e0d541d3f0db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:34:02.162735 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d0d29-8ea0-42be-8f59-e0d541d3f0db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:02.164310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d0d29-8ea0-42be-8f59-e0d541d3f0db admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:02.176195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1052/2086] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:34:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:34:02.180378 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a674441-6cfc-4137-95d4-9b0a45bf9c3f None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:02.189787 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a674441-6cfc-4137-95d4-9b0a45bf9c3f None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:02.194527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3cad3a-8a56-4e39-aef1-e9dedb2ca843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:02.195128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3cad3a-8a56-4e39-aef1-e9dedb2ca843 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:02.195407 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3cad3a-8a56-4e39-aef1-e9dedb2ca843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:02.195673 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3cad3a-8a56-4e39-aef1-e9dedb2ca843 admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:02.196311 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3cad3a-8a56-4e39-aef1-e9dedb2ca843 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:02.199101 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3cad3a-8a56-4e39-aef1-e9dedb2ca843 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:02.230128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1035/2087] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:34:02 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:34:02.231706 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d0d29-8ea0-42be-8f59-e0d541d3f0db admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:02.241774 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d0d29-8ea0-42be-8f59-e0d541d3f0db admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:02.300475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97dc5b-907a-47b2-a212-d6ab4096d2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:02.301088 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97dc5b-907a-47b2-a212-d6ab4096d2bf admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:02.301367 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97dc5b-907a-47b2-a212-d6ab4096d2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:02.301670 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97dc5b-907a-47b2-a212-d6ab4096d2bf admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:02.302351 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97dc5b-907a-47b2-a212-d6ab4096d2bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:02.304260 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97dc5b-907a-47b2-a212-d6ab4096d2bf admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:02.330351 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1053/2088] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:34:02 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:34:02.331613 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3cad3a-8a56-4e39-aef1-e9dedb2ca843 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:02.342916 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3cad3a-8a56-4e39-aef1-e9dedb2ca843 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:02.347765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b6bac-49e8-4b74-9ab2-4178da029664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:02.348361 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b6bac-49e8-4b74-9ab2-4178da029664 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:02.348640 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b6bac-49e8-4b74-9ab2-4178da029664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:02.348911 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b6bac-49e8-4b74-9ab2-4178da029664 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:02.349808 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7b6bac-49e8-4b74-9ab2-4178da029664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:02.351431 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7b6bac-49e8-4b74-9ab2-4178da029664 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:02.362508 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1036/2089] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:34:02.366104 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97dc5b-907a-47b2-a212-d6ab4096d2bf admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:02.376923 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97dc5b-907a-47b2-a212-d6ab4096d2bf admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:02.381529 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93dce94-8cf5-4c96-83b8-da54c0571b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:02.382159 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93dce94-8cf5-4c96-83b8-da54c0571b6e admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:02.382436 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93dce94-8cf5-4c96-83b8-da54c0571b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:02.382709 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93dce94-8cf5-4c96-83b8-da54c0571b6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:02.383343 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93dce94-8cf5-4c96-83b8-da54c0571b6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:02.384925 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93dce94-8cf5-4c96-83b8-da54c0571b6e admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:02.395387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1054/2090] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:34:02.399436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b6bac-49e8-4b74-9ab2-4178da029664 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:02.405972 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:02.409815 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b6bac-49e8-4b74-9ab2-4178da029664 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:02.416355 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064623c8-56aa-4f98-b80f-b7c0d5c6d566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:02.417783 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064623c8-56aa-4f98-b80f-b7c0d5c6d566 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:02.418313 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064623c8-56aa-4f98-b80f-b7c0d5c6d566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:02.418670 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064623c8-56aa-4f98-b80f-b7c0d5c6d566 admin admin] PATH_INFO: `/v3/projects/ed4a241a593543d69cdd1fcfb757419c/users/2360ae2b47f64ee58642ba8253d81aa3/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:02.434563 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064623c8-56aa-4f98-b80f-b7c0d5c6d566 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed4a241a593543d69cdd1fcfb757419c, user_id=2360ae2b47f64ee58642ba8253d81aa3, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:02.437322 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064623c8-56aa-4f98-b80f-b7c0d5c6d566 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:02.449664 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1037/2091] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:34:02 2022] PUT /identity/v3/projects/ed4a241a593543d69cdd1fcfb757419c/users/2360ae2b47f64ee58642ba8253d81aa3/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:02.460500 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499cfae-bb8c-4889-b71f-7d0f0a416f06 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:02.460780 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499cfae-bb8c-4889-b71f-7d0f0a416f06 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:02.461076 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499cfae-bb8c-4889-b71f-7d0f0a416f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:02.488708 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-9499cfae-bb8c-4889-b71f-7d0f0a416f06 None None] MFA Rules not processed for user `2360ae2b47f64ee58642ba8253d81aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:02.489910 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-9499cfae-bb8c-4889-b71f-7d0f0a416f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:02.555678 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1055/2092] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:34:02.567189 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064623c8-56aa-4f98-b80f-b7c0d5c6d566 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:02.568950 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-064623c8-56aa-4f98-b80f-b7c0d5c6d566 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:02.626088 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064623c8-56aa-4f98-b80f-b7c0d5c6d566 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:02.629720 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b2b2c-92a3-4b55-a106-8e944219c1ef service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41b242fbe2504f8cac26ce4ff46f5390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:02.630344 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b2b2c-92a3-4b55-a106-8e944219c1ef service glance] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:02.630626 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b2b2c-92a3-4b55-a106-8e944219c1ef service glance] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:02.630900 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b2b2c-92a3-4b55-a106-8e944219c1ef service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:02.639188 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b2b2c-92a3-4b55-a106-8e944219c1ef service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:02.640611 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b2b2c-92a3-4b55-a106-8e944219c1ef service glance] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:02.649350 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1038/2093] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:34:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:34:05.578140 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499cfae-bb8c-4889-b71f-7d0f0a416f06 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:05.580223 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-9499cfae-bb8c-4889-b71f-7d0f0a416f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:05.594861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:05.651565 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499cfae-bb8c-4889-b71f-7d0f0a416f06 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:05.655297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4942f6e3-a7f0-445a-89f7-ce74ae4284e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:05.655902 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942f6e3-a7f0-445a-89f7-ce74ae4284e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:05.656176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942f6e3-a7f0-445a-89f7-ce74ae4284e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:05.656444 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942f6e3-a7f0-445a-89f7-ce74ae4284e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:05.658257 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-4942f6e3-a7f0-445a-89f7-ce74ae4284e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:05.755413 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4942f6e3-a7f0-445a-89f7-ce74ae4284e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:05.757027 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4942f6e3-a7f0-445a-89f7-ce74ae4284e4 service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:05.771339 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1056/2094] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:34:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:34:05.810363 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b2b2c-92a3-4b55-a106-8e944219c1ef service glance] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:05.813127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-3c5b2b2c-92a3-4b55-a106-8e944219c1ef service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:05.826538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b2b2c-92a3-4b55-a106-8e944219c1ef service glance] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:05.830623 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b12665-cc89-4715-843f-1ee64d80318d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:05.831243 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b12665-cc89-4715-843f-1ee64d80318d service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:05.831516 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b12665-cc89-4715-843f-1ee64d80318d service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:05.831782 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b12665-cc89-4715-843f-1ee64d80318d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:05.833647 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-b2b12665-cc89-4715-843f-1ee64d80318d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:05.910766 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b12665-cc89-4715-843f-1ee64d80318d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:05.912304 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b12665-cc89-4715-843f-1ee64d80318d service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:05.923784 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1039/2095] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:34:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:34:06.630885 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4942f6e3-a7f0-445a-89f7-ce74ae4284e4 service cinder] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:06.640343 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4942f6e3-a7f0-445a-89f7-ce74ae4284e4 service cinder] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:06.644250 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ee24f-9f09-4898-9e72-3f8cace35718 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:06.644856 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ee24f-9f09-4898-9e72-3f8cace35718 service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:06.645189 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ee24f-9f09-4898-9e72-3f8cace35718 service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:06.645466 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ee24f-9f09-4898-9e72-3f8cace35718 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:06.647204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-059ee24f-9f09-4898-9e72-3f8cace35718 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:06.725344 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059ee24f-9f09-4898-9e72-3f8cace35718 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:06.726968 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059ee24f-9f09-4898-9e72-3f8cace35718 service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:06.741640 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1057/2096] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:34:06 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:34:08.521740 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b12665-cc89-4715-843f-1ee64d80318d service cinder] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:08.524254 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-b2b12665-cc89-4715-843f-1ee64d80318d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:08.609198 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b12665-cc89-4715-843f-1ee64d80318d service cinder] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:08.614212 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8808cda0-2cee-4908-b711-fbb324c2a577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:08.614967 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808cda0-2cee-4908-b711-fbb324c2a577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:08.615269 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808cda0-2cee-4908-b711-fbb324c2a577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:08.615816 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808cda0-2cee-4908-b711-fbb324c2a577 admin admin] PATH_INFO: `/v3/users/29772aaeb43b4386840dee4491fe71d6` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:08.617826 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8808cda0-2cee-4908-b711-fbb324c2a577 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29772aaeb43b4386840dee4491fe71d6)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:08.620580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8808cda0-2cee-4908-b711-fbb324c2a577 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:08.644797 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:08.671515 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-8808cda0-2cee-4908-b711-fbb324c2a577 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29772aaeb43b4386840dee4491fe71d6'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:08.672178 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-8808cda0-2cee-4908-b711-fbb324c2a577 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29772aaeb43b4386840dee4491fe71d6'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:08.672785 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-8808cda0-2cee-4908-b711-fbb324c2a577 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29772aaeb43b4386840dee4491fe71d6'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:08.680241 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-8808cda0-2cee-4908-b711-fbb324c2a577 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29772aaeb43b4386840dee4491fe71d6'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:08.692851 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-8808cda0-2cee-4908-b711-fbb324c2a577 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29772aaeb43b4386840dee4491fe71d6'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:08.710306 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1040/2097] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:34:08 2022] DELETE /identity/v3/users/29772aaeb43b4386840dee4491fe71d6 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:08.715274 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509fc29-6995-43cb-986f-99dbf1cd5a15 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:08.715756 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509fc29-6995-43cb-986f-99dbf1cd5a15 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:08.715933 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509fc29-6995-43cb-986f-99dbf1cd5a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:08.736823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-5509fc29-6995-43cb-986f-99dbf1cd5a15 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:08.738036 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-5509fc29-6995-43cb-986f-99dbf1cd5a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:08.807800 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:08.813359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1058/2098] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:34:08.823132 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8808cda0-2cee-4908-b711-fbb324c2a577 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:08.824738 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-8808cda0-2cee-4908-b711-fbb324c2a577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:08.887318 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8808cda0-2cee-4908-b711-fbb324c2a577 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:08.890890 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb1f9e3-c761-44a2-bb74-494c1d051aa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:08.891503 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb1f9e3-c761-44a2-bb74-494c1d051aa2 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:08.891780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb1f9e3-c761-44a2-bb74-494c1d051aa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:08.892054 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb1f9e3-c761-44a2-bb74-494c1d051aa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:08.901781 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb1f9e3-c761-44a2-bb74-494c1d051aa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:08.903253 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb1f9e3-c761-44a2-bb74-494c1d051aa2 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:08.913209 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1041/2099] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:34:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:34:09.197787 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509fc29-6995-43cb-986f-99dbf1cd5a15 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:09.202011 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-5509fc29-6995-43cb-986f-99dbf1cd5a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:09.218973 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509fc29-6995-43cb-986f-99dbf1cd5a15 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:09.225310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743f56f-c940-49ee-9780-7707b228187c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:09.225995 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743f56f-c940-49ee-9780-7707b228187c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:09.226383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743f56f-c940-49ee-9780-7707b228187c admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:09.226658 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743f56f-c940-49ee-9780-7707b228187c admin admin] PATH_INFO: `/v3/projects/4ca06d599bbd4e1fb0b18fb3b2abc99e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:09.227815 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743f56f-c940-49ee-9780-7707b228187c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ca06d599bbd4e1fb0b18fb3b2abc99e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:09.229748 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743f56f-c940-49ee-9780-7707b228187c admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:09.284143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7743f56f-c940-49ee-9780-7707b228187c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ca06d599bbd4e1fb0b18fb3b2abc99e', 'request_id': 'req-7743f56f-c940-49ee-9780-7707b228187c'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:09.290829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7743f56f-c940-49ee-9780-7707b228187c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ca06d599bbd4e1fb0b18fb3b2abc99e', 'request_id': 'req-7743f56f-c940-49ee-9780-7707b228187c'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:09.299062 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7743f56f-c940-49ee-9780-7707b228187c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ca06d599bbd4e1fb0b18fb3b2abc99e', 'request_id': 'req-7743f56f-c940-49ee-9780-7707b228187c'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:09.310845 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7743f56f-c940-49ee-9780-7707b228187c admin admin] The token cache is being invalidate because project 4ca06d599bbd4e1fb0b18fb3b2abc99e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:34:09.311399 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7743f56f-c940-49ee-9780-7707b228187c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:09.312265 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7743f56f-c940-49ee-9780-7707b228187c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:09.313992 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1059/2100] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:34:09 2022] DELETE /identity/v3/projects/4ca06d599bbd4e1fb0b18fb3b2abc99e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:09.318208 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb1f9e3-c761-44a2-bb74-494c1d051aa2 service neutron] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:09.329128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb1f9e3-c761-44a2-bb74-494c1d051aa2 service neutron] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:09.423214 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff6807-2b90-4a3c-b61b-4b300aa02cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:09.423930 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff6807-2b90-4a3c-b61b-4b300aa02cde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:09.424277 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff6807-2b90-4a3c-b61b-4b300aa02cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:09.424618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff6807-2b90-4a3c-b61b-4b300aa02cde admin admin] PATH_INFO: `/v3/users/6a840e59d47b4e48b3f58fe736647ad0` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:09.425894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bff6807-2b90-4a3c-b61b-4b300aa02cde admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a840e59d47b4e48b3f58fe736647ad0)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:09.428509 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bff6807-2b90-4a3c-b61b-4b300aa02cde admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:09.498437 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:09.500986 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-0bff6807-2b90-4a3c-b61b-4b300aa02cde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a840e59d47b4e48b3f58fe736647ad0'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:09.502320 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-0bff6807-2b90-4a3c-b61b-4b300aa02cde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a840e59d47b4e48b3f58fe736647ad0'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:09.503727 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-0bff6807-2b90-4a3c-b61b-4b300aa02cde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a840e59d47b4e48b3f58fe736647ad0'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:09.515148 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-0bff6807-2b90-4a3c-b61b-4b300aa02cde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a840e59d47b4e48b3f58fe736647ad0'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:09.530534 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-0bff6807-2b90-4a3c-b61b-4b300aa02cde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a840e59d47b4e48b3f58fe736647ad0'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:09.546705 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1042/2101] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:34:09 2022] DELETE /identity/v3/users/6a840e59d47b4e48b3f58fe736647ad0 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:10.155915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743f56f-c940-49ee-9780-7707b228187c admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:10.157549 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-7743f56f-c940-49ee-9780-7707b228187c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:10.170988 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743f56f-c940-49ee-9780-7707b228187c admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:10.240290 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3e6bc1-39ab-4113-9a7c-27a2f7dd53e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:10.240937 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e6bc1-39ab-4113-9a7c-27a2f7dd53e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:10.241286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e6bc1-39ab-4113-9a7c-27a2f7dd53e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:10.241638 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e6bc1-39ab-4113-9a7c-27a2f7dd53e6 admin admin] PATH_INFO: `/v3/projects/793a2228f8d647779cb892e54164a073` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:10.242886 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3e6bc1-39ab-4113-9a7c-27a2f7dd53e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=793a2228f8d647779cb892e54164a073)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:10.245414 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3e6bc1-39ab-4113-9a7c-27a2f7dd53e6 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:10.298600 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7e3e6bc1-39ab-4113-9a7c-27a2f7dd53e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '793a2228f8d647779cb892e54164a073', 'request_id': 'req-7e3e6bc1-39ab-4113-9a7c-27a2f7dd53e6'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:10.305203 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7e3e6bc1-39ab-4113-9a7c-27a2f7dd53e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '793a2228f8d647779cb892e54164a073', 'request_id': 'req-7e3e6bc1-39ab-4113-9a7c-27a2f7dd53e6'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:10.313659 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7e3e6bc1-39ab-4113-9a7c-27a2f7dd53e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '793a2228f8d647779cb892e54164a073', 'request_id': 'req-7e3e6bc1-39ab-4113-9a7c-27a2f7dd53e6'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:10.327073 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7e3e6bc1-39ab-4113-9a7c-27a2f7dd53e6 admin admin] The token cache is being invalidate because project 793a2228f8d647779cb892e54164a073 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:34:10.327368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7e3e6bc1-39ab-4113-9a7c-27a2f7dd53e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:10.328582 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7e3e6bc1-39ab-4113-9a7c-27a2f7dd53e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:10.330439 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1060/2102] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:34:10 2022] DELETE /identity/v3/projects/793a2228f8d647779cb892e54164a073 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:10.334612 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff6807-2b90-4a3c-b61b-4b300aa02cde admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:10.335683 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-0bff6807-2b90-4a3c-b61b-4b300aa02cde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:10.354705 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff6807-2b90-4a3c-b61b-4b300aa02cde admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:10.431234 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1279a-3993-42ca-80ba-18b9ae826dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:10.431871 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1279a-3993-42ca-80ba-18b9ae826dd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:10.432149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1279a-3993-42ca-80ba-18b9ae826dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:10.433042 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1279a-3993-42ca-80ba-18b9ae826dd0 admin admin] PATH_INFO: `/v3/users/3325b1e1884641c4b036bc4d049f0f9e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:10.434208 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1279a-3993-42ca-80ba-18b9ae826dd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3325b1e1884641c4b036bc4d049f0f9e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:10.436614 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1279a-3993-42ca-80ba-18b9ae826dd0 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:10.504923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-20a1279a-3993-42ca-80ba-18b9ae826dd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3325b1e1884641c4b036bc4d049f0f9e'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:10.505884 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-20a1279a-3993-42ca-80ba-18b9ae826dd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3325b1e1884641c4b036bc4d049f0f9e'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:10.506850 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-20a1279a-3993-42ca-80ba-18b9ae826dd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3325b1e1884641c4b036bc4d049f0f9e'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:10.521649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-20a1279a-3993-42ca-80ba-18b9ae826dd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3325b1e1884641c4b036bc4d049f0f9e'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:10.549171 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-20a1279a-3993-42ca-80ba-18b9ae826dd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3325b1e1884641c4b036bc4d049f0f9e'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:10.569135 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1043/2103] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:34:10 2022] DELETE /identity/v3/users/3325b1e1884641c4b036bc4d049f0f9e => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:10.975281 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3e6bc1-39ab-4113-9a7c-27a2f7dd53e6 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:10.976881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-7e3e6bc1-39ab-4113-9a7c-27a2f7dd53e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:11.000451 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3e6bc1-39ab-4113-9a7c-27a2f7dd53e6 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:11.019880 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:11.056174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1279a-3993-42ca-80ba-18b9ae826dd0 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:11.060167 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-20a1279a-3993-42ca-80ba-18b9ae826dd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:11.087206 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4aba7-15a5-4aa3-bba4-e84c3b5f3dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:11.087846 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4aba7-15a5-4aa3-bba4-e84c3b5f3dc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:11.088254 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4aba7-15a5-4aa3-bba4-e84c3b5f3dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:11.088988 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4aba7-15a5-4aa3-bba4-e84c3b5f3dc9 admin admin] PATH_INFO: `/v3/projects/78d9497c839f4002aa630e7ce49e3c34` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:11.097719 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d4aba7-15a5-4aa3-bba4-e84c3b5f3dc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78d9497c839f4002aa630e7ce49e3c34)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:11.097719 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d4aba7-15a5-4aa3-bba4-e84c3b5f3dc9 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:11.140963 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:11.168720 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1279a-3993-42ca-80ba-18b9ae826dd0 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:11.173355 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-b5d4aba7-15a5-4aa3-bba4-e84c3b5f3dc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78d9497c839f4002aa630e7ce49e3c34', 'request_id': 'req-b5d4aba7-15a5-4aa3-bba4-e84c3b5f3dc9'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:11.181766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-b5d4aba7-15a5-4aa3-bba4-e84c3b5f3dc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78d9497c839f4002aa630e7ce49e3c34', 'request_id': 'req-b5d4aba7-15a5-4aa3-bba4-e84c3b5f3dc9'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:11.193909 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-b5d4aba7-15a5-4aa3-bba4-e84c3b5f3dc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78d9497c839f4002aa630e7ce49e3c34', 'request_id': 'req-b5d4aba7-15a5-4aa3-bba4-e84c3b5f3dc9'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:11.214821 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-b5d4aba7-15a5-4aa3-bba4-e84c3b5f3dc9 admin admin] The token cache is being invalidate because project 78d9497c839f4002aa630e7ce49e3c34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:34:11.215119 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-b5d4aba7-15a5-4aa3-bba4-e84c3b5f3dc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:11.216024 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-b5d4aba7-15a5-4aa3-bba4-e84c3b5f3dc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:11.217835 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1061/2104] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:34:10 2022] DELETE /identity/v3/projects/78d9497c839f4002aa630e7ce49e3c34 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:11.225862 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1b351-cb29-43cd-8073-b6f9304047bd None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:11.226065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1b351-cb29-43cd-8073-b6f9304047bd None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:11.226767 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1b351-cb29-43cd-8073-b6f9304047bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:11.248970 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-98d1b351-cb29-43cd-8073-b6f9304047bd None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:11.250168 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-98d1b351-cb29-43cd-8073-b6f9304047bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:11.265511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8408d0e-04b0-490f-9954-030328b97a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:11.266563 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8408d0e-04b0-490f-9954-030328b97a7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:11.266911 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8408d0e-04b0-490f-9954-030328b97a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:11.267255 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8408d0e-04b0-490f-9954-030328b97a7d admin admin] PATH_INFO: `/v3/users/6eef257db6c6436ea91fcf267632e67a` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:11.268475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8408d0e-04b0-490f-9954-030328b97a7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6eef257db6c6436ea91fcf267632e67a)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:11.270519 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8408d0e-04b0-490f-9954-030328b97a7d admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:11.344119 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1062/2105] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:34:11.345750 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aaf015-5965-4c40-a115-c62f736f4ca1 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:11.346102 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aaf015-5965-4c40-a115-c62f736f4ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:11.346398 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aaf015-5965-4c40-a115-c62f736f4ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:11.350983 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-a8408d0e-04b0-490f-9954-030328b97a7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6eef257db6c6436ea91fcf267632e67a'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:11.351654 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-a8408d0e-04b0-490f-9954-030328b97a7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6eef257db6c6436ea91fcf267632e67a'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:11.352323 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-a8408d0e-04b0-490f-9954-030328b97a7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6eef257db6c6436ea91fcf267632e67a'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:11.366401 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-a8408d0e-04b0-490f-9954-030328b97a7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6eef257db6c6436ea91fcf267632e67a'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:11.375486 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-a9aaf015-5965-4c40-a115-c62f736f4ca1 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:11.376740 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-a9aaf015-5965-4c40-a115-c62f736f4ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:11.381608 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-a8408d0e-04b0-490f-9954-030328b97a7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6eef257db6c6436ea91fcf267632e67a'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:11.386560 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1063/2106] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:34:11.391158 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aaf015-5965-4c40-a115-c62f736f4ca1 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:11.405058 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aaf015-5965-4c40-a115-c62f736f4ca1 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:11.406009 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1044/2107] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:34:11 2022] DELETE /identity/v3/users/6eef257db6c6436ea91fcf267632e67a => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:11.410624 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032038e6-29cc-45fa-b815-bc0bdc3c3c18 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:11.410909 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032038e6-29cc-45fa-b815-bc0bdc3c3c18 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:11.411425 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032038e6-29cc-45fa-b815-bc0bdc3c3c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:11.442049 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-032038e6-29cc-45fa-b815-bc0bdc3c3c18 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:11.443324 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-032038e6-29cc-45fa-b815-bc0bdc3c3c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:11.454442 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:11.522885 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f3a68-c5f9-4ee6-969a-4b6ff384d33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:11.524165 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f3a68-c5f9-4ee6-969a-4b6ff384d33f admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:11.524623 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f3a68-c5f9-4ee6-969a-4b6ff384d33f admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:11.525069 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f3a68-c5f9-4ee6-969a-4b6ff384d33f admin admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:11.525907 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f3a68-c5f9-4ee6-969a-4b6ff384d33f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:34:11.527299 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f3a68-c5f9-4ee6-969a-4b6ff384d33f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:11.529913 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f3a68-c5f9-4ee6-969a-4b6ff384d33f admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:11.545279 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1064/2108] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:34:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:34:11.545669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1045/2109] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:34:11.551962 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f3a68-c5f9-4ee6-969a-4b6ff384d33f admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:11.555755 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032038e6-29cc-45fa-b815-bc0bdc3c3c18 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:11.557297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-032038e6-29cc-45fa-b815-bc0bdc3c3c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:11.563473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f3a68-c5f9-4ee6-969a-4b6ff384d33f admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:11.568501 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:11.570383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb061807-7c25-45b0-abe6-49766478520f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:11.571100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb061807-7c25-45b0-abe6-49766478520f admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:11.571444 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb061807-7c25-45b0-abe6-49766478520f admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:11.572076 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb061807-7c25-45b0-abe6-49766478520f admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:11.572776 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb061807-7c25-45b0-abe6-49766478520f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:11.575021 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb061807-7c25-45b0-abe6-49766478520f admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:11.610274 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1065/2110] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:34:11 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:34:11.611014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb061807-7c25-45b0-abe6-49766478520f admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:11.629994 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032038e6-29cc-45fa-b815-bc0bdc3c3c18 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:11.679087 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb061807-7c25-45b0-abe6-49766478520f admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:11.684996 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e10df-466a-4553-a38a-96e4f907a094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:11.685759 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e10df-466a-4553-a38a-96e4f907a094 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:11.686085 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e10df-466a-4553-a38a-96e4f907a094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:11.686465 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e10df-466a-4553-a38a-96e4f907a094 admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:11.687106 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e10df-466a-4553-a38a-96e4f907a094 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:11.690709 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e10df-466a-4553-a38a-96e4f907a094 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:11.698856 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad6975-5419-46ab-a858-f97d4650b481 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:11.699468 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad6975-5419-46ab-a858-f97d4650b481 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:11.699749 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad6975-5419-46ab-a858-f97d4650b481 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:11.700131 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad6975-5419-46ab-a858-f97d4650b481 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:11.715591 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad6975-5419-46ab-a858-f97d4650b481 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:11.715591 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad6975-5419-46ab-a858-f97d4650b481 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:11.720374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1066/2111] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:34:11 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:34:11.724548 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e10df-466a-4553-a38a-96e4f907a094 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:11.733487 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1046/2112] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:34:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:34:11.736555 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e10df-466a-4553-a38a-96e4f907a094 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:11.743063 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30924b0a-551d-42fe-963f-520ea4adf1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:11.743674 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30924b0a-551d-42fe-963f-520ea4adf1a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:11.743950 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30924b0a-551d-42fe-963f-520ea4adf1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:11.744262 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30924b0a-551d-42fe-963f-520ea4adf1a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:11.745056 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30924b0a-551d-42fe-963f-520ea4adf1a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:11.749468 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30924b0a-551d-42fe-963f-520ea4adf1a6 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:11.761886 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad6975-5419-46ab-a858-f97d4650b481 service neutron] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:11.763468 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-f8ad6975-5419-46ab-a858-f97d4650b481 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:11.767901 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1067/2113] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:34:11.768792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30924b0a-551d-42fe-963f-520ea4adf1a6 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:11.782610 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30924b0a-551d-42fe-963f-520ea4adf1a6 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:11.787904 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad6975-5419-46ab-a858-f97d4650b481 service neutron] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:11.794584 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b51350-9fd2-4e10-b0ad-d87218a0584f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:11.795151 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b51350-9fd2-4e10-b0ad-d87218a0584f admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:11.795438 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b51350-9fd2-4e10-b0ad-d87218a0584f admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:11.796150 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b51350-9fd2-4e10-b0ad-d87218a0584f admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:11.797067 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b51350-9fd2-4e10-b0ad-d87218a0584f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:11.805086 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398151d7-60e8-40b7-b0bf-3fb0536594ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:11.805086 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398151d7-60e8-40b7-b0bf-3fb0536594ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:11.805340 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b51350-9fd2-4e10-b0ad-d87218a0584f admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:11.805417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398151d7-60e8-40b7-b0bf-3fb0536594ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:11.805417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398151d7-60e8-40b7-b0bf-3fb0536594ad admin admin] PATH_INFO: `/v3/users/5762dd01182d4c2a8856591ef56f7ae7` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:11.805417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398151d7-60e8-40b7-b0bf-3fb0536594ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=5762dd01182d4c2a8856591ef56f7ae7)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:11.805854 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398151d7-60e8-40b7-b0bf-3fb0536594ad admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:11.839835 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1068/2114] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:34:11.840622 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b51350-9fd2-4e10-b0ad-d87218a0584f admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:11.875470 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b51350-9fd2-4e10-b0ad-d87218a0584f admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:11.896334 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa66bc-b257-42a1-9c02-d970901a88ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:11.898220 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa66bc-b257-42a1-9c02-d970901a88ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:11.899556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa66bc-b257-42a1-9c02-d970901a88ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:11.900264 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa66bc-b257-42a1-9c02-d970901a88ff admin admin] PATH_INFO: `/v3/projects/688e567df52e44519e1e79da7b8b4d4e/users/a8f7651292da480496db678f4269a249/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:11.905654 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-398151d7-60e8-40b7-b0bf-3fb0536594ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5762dd01182d4c2a8856591ef56f7ae7'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:11.905792 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-398151d7-60e8-40b7-b0bf-3fb0536594ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5762dd01182d4c2a8856591ef56f7ae7'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:11.907320 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-398151d7-60e8-40b7-b0bf-3fb0536594ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5762dd01182d4c2a8856591ef56f7ae7'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:11.924818 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa66bc-b257-42a1-9c02-d970901a88ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=688e567df52e44519e1e79da7b8b4d4e, user_id=a8f7651292da480496db678f4269a249, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:11.928020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa66bc-b257-42a1-9c02-d970901a88ff admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:11.934770 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-398151d7-60e8-40b7-b0bf-3fb0536594ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5762dd01182d4c2a8856591ef56f7ae7'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:11.965118 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1069/2115] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:34:11 2022] PUT /identity/v3/projects/688e567df52e44519e1e79da7b8b4d4e/users/a8f7651292da480496db678f4269a249/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:11.977697 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-398151d7-60e8-40b7-b0bf-3fb0536594ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5762dd01182d4c2a8856591ef56f7ae7'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:11.982413 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d046934-83ef-44f5-9cae-f9db159ea9d1 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:11.983157 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d046934-83ef-44f5-9cae-f9db159ea9d1 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:11.983624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d046934-83ef-44f5-9cae-f9db159ea9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:12.008208 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1047/2116] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:34:11 2022] DELETE /identity/v3/users/5762dd01182d4c2a8856591ef56f7ae7 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:12.012995 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235d48b-da81-4d1c-9961-6f1ae27785a3 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:12.013275 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235d48b-da81-4d1c-9961-6f1ae27785a3 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:12.013551 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235d48b-da81-4d1c-9961-6f1ae27785a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:12.043550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-2d046934-83ef-44f5-9cae-f9db159ea9d1 None None] MFA Rules not processed for user `a8f7651292da480496db678f4269a249`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:12.044836 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-2d046934-83ef-44f5-9cae-f9db159ea9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:12.046456 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-1235d48b-da81-4d1c-9961-6f1ae27785a3 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:12.048004 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-1235d48b-da81-4d1c-9961-6f1ae27785a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:12.104695 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:12.115533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:12.159336 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1048/2117] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:34:12.169710 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1070/2118] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:34:12.174701 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235d48b-da81-4d1c-9961-6f1ae27785a3 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:12.176961 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-1235d48b-da81-4d1c-9961-6f1ae27785a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:12.190406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d046934-83ef-44f5-9cae-f9db159ea9d1 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:12.194194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-2d046934-83ef-44f5-9cae-f9db159ea9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:12.214968 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235d48b-da81-4d1c-9961-6f1ae27785a3 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:12.231126 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe688e8d-183c-4fda-8f03-65b49f5ab07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:12.232548 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe688e8d-183c-4fda-8f03-65b49f5ab07b admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:12.232914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe688e8d-183c-4fda-8f03-65b49f5ab07b admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:12.233264 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe688e8d-183c-4fda-8f03-65b49f5ab07b admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:12.234642 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe688e8d-183c-4fda-8f03-65b49f5ab07b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:12.237186 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe688e8d-183c-4fda-8f03-65b49f5ab07b admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:12.309293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1049/2119] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:34:12 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:34:12.310149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe688e8d-183c-4fda-8f03-65b49f5ab07b admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:12.371540 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d046934-83ef-44f5-9cae-f9db159ea9d1 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:12.428508 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe688e8d-183c-4fda-8f03-65b49f5ab07b admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:12.435218 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb161770-4f07-400c-b0aa-a17dbf80b841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:12.435975 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb161770-4f07-400c-b0aa-a17dbf80b841 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:12.436326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb161770-4f07-400c-b0aa-a17dbf80b841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:12.436676 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb161770-4f07-400c-b0aa-a17dbf80b841 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:12.437465 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb161770-4f07-400c-b0aa-a17dbf80b841 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:12.440810 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb161770-4f07-400c-b0aa-a17dbf80b841 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:12.463343 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1fb4c1-cd31-48cd-a1fb-c24efb219e19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:12.464174 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fb4c1-cd31-48cd-a1fb-c24efb219e19 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:12.464850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fb4c1-cd31-48cd-a1fb-c24efb219e19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:12.465189 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fb4c1-cd31-48cd-a1fb-c24efb219e19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:12.475593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1050/2120] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:34:12 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:34:12.476670 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb161770-4f07-400c-b0aa-a17dbf80b841 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:12.479995 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-cb161770-4f07-400c-b0aa-a17dbf80b841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:12.491368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1fb4c1-cd31-48cd-a1fb-c24efb219e19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:12.494314 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1fb4c1-cd31-48cd-a1fb-c24efb219e19 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:12.501120 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb161770-4f07-400c-b0aa-a17dbf80b841 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:12.508679 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ebb6a-5b66-4a53-a42c-0b05f40be90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:12.512910 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1071/2121] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:34:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:34:12.514520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1fb4c1-cd31-48cd-a1fb-c24efb219e19 service neutron] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:12.515606 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ebb6a-5b66-4a53-a42c-0b05f40be90c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:12.516107 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ebb6a-5b66-4a53-a42c-0b05f40be90c admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:12.516408 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ebb6a-5b66-4a53-a42c-0b05f40be90c admin admin] PATH_INFO: `/v3/projects/42e1027669fb454899c6a54c410d73ed` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:12.518206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ebb6a-5b66-4a53-a42c-0b05f40be90c admin admin] RBAC: Authorizing `identity:delete_project(project_id=42e1027669fb454899c6a54c410d73ed)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:12.520675 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ebb6a-5b66-4a53-a42c-0b05f40be90c admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:12.533063 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1fb4c1-cd31-48cd-a1fb-c24efb219e19 service neutron] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:12.541060 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7accb9-38df-4562-8595-dbba77563142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:12.541796 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7accb9-38df-4562-8595-dbba77563142 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:12.542105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7accb9-38df-4562-8595-dbba77563142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:12.542770 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7accb9-38df-4562-8595-dbba77563142 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:12.543169 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7accb9-38df-4562-8595-dbba77563142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:12.545730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7accb9-38df-4562-8595-dbba77563142 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:12.563661 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1072/2122] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:34:12.567810 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7accb9-38df-4562-8595-dbba77563142 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:12.588103 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7accb9-38df-4562-8595-dbba77563142 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:12.622437 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-063ebb6a-5b66-4a53-a42c-0b05f40be90c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42e1027669fb454899c6a54c410d73ed', 'request_id': 'req-063ebb6a-5b66-4a53-a42c-0b05f40be90c'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:12.643038 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-063ebb6a-5b66-4a53-a42c-0b05f40be90c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42e1027669fb454899c6a54c410d73ed', 'request_id': 'req-063ebb6a-5b66-4a53-a42c-0b05f40be90c'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:12.656974 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-063ebb6a-5b66-4a53-a42c-0b05f40be90c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42e1027669fb454899c6a54c410d73ed', 'request_id': 'req-063ebb6a-5b66-4a53-a42c-0b05f40be90c'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:12.689766 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-063ebb6a-5b66-4a53-a42c-0b05f40be90c admin admin] The token cache is being invalidate because project 42e1027669fb454899c6a54c410d73ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:34:12.690938 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-063ebb6a-5b66-4a53-a42c-0b05f40be90c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:12.691804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-063ebb6a-5b66-4a53-a42c-0b05f40be90c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:12.694271 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1051/2123] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:34:12 2022] DELETE /identity/v3/projects/42e1027669fb454899c6a54c410d73ed => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:12.698236 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ebb6a-5b66-4a53-a42c-0b05f40be90c admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:12.700209 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-063ebb6a-5b66-4a53-a42c-0b05f40be90c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:12.712953 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e02629-8981-4b30-a6ed-1a28534e0a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:12.713634 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e02629-8981-4b30-a6ed-1a28534e0a5e admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:12.714062 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e02629-8981-4b30-a6ed-1a28534e0a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:12.714211 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e02629-8981-4b30-a6ed-1a28534e0a5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:12.714873 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e02629-8981-4b30-a6ed-1a28534e0a5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:12.716624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e02629-8981-4b30-a6ed-1a28534e0a5e admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:12.742777 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1073/2124] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:34:12.743747 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e02629-8981-4b30-a6ed-1a28534e0a5e admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:12.753649 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-c9e02629-8981-4b30-a6ed-1a28534e0a5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:12.773500 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:12.778947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e02629-8981-4b30-a6ed-1a28534e0a5e admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:12.784784 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23fe95-e0b2-4418-946b-ca88b17efb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:12.785527 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23fe95-e0b2-4418-946b-ca88b17efb7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:12.785749 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23fe95-e0b2-4418-946b-ca88b17efb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:12.786049 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23fe95-e0b2-4418-946b-ca88b17efb7a admin admin] PATH_INFO: `/v3/projects/1644ce5f4f474a3e9ea5058ddc55ef14/users/d013ac4b40e74366b26980fc7ec4369b/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:12.819487 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ebb6a-5b66-4a53-a42c-0b05f40be90c admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:12.826397 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5300e12-137b-48b6-a3c5-56bd6b88792a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:12.827135 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5300e12-137b-48b6-a3c5-56bd6b88792a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:12.827488 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5300e12-137b-48b6-a3c5-56bd6b88792a admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:12.828174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5300e12-137b-48b6-a3c5-56bd6b88792a admin admin] PATH_INFO: `/v3/users/6bbe23a7b451494e954fd47eab70b644` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:12.829046 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5300e12-137b-48b6-a3c5-56bd6b88792a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bbe23a7b451494e954fd47eab70b644)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:12.831314 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5300e12-137b-48b6-a3c5-56bd6b88792a admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:12.855050 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf23fe95-e0b2-4418-946b-ca88b17efb7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1644ce5f4f474a3e9ea5058ddc55ef14, user_id=d013ac4b40e74366b26980fc7ec4369b, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:12.858444 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf23fe95-e0b2-4418-946b-ca88b17efb7a admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:12.878495 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1074/2125] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:34:12 2022] PUT /identity/v3/projects/1644ce5f4f474a3e9ea5058ddc55ef14/users/d013ac4b40e74366b26980fc7ec4369b/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:12.891715 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b6c4f-f659-4806-82f4-5598afee8c6d None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:12.891929 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b6c4f-f659-4806-82f4-5598afee8c6d None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:12.891929 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b6c4f-f659-4806-82f4-5598afee8c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:12.907336 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:12.913884 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-d5300e12-137b-48b6-a3c5-56bd6b88792a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bbe23a7b451494e954fd47eab70b644'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:12.916071 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-d5300e12-137b-48b6-a3c5-56bd6b88792a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bbe23a7b451494e954fd47eab70b644'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:12.918593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-d5300e12-137b-48b6-a3c5-56bd6b88792a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bbe23a7b451494e954fd47eab70b644'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:12.930822 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-d5300e12-137b-48b6-a3c5-56bd6b88792a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bbe23a7b451494e954fd47eab70b644'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:12.935489 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-cc8b6c4f-f659-4806-82f4-5598afee8c6d None None] MFA Rules not processed for user `d013ac4b40e74366b26980fc7ec4369b`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:12.936761 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-cc8b6c4f-f659-4806-82f4-5598afee8c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:12.946959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-d5300e12-137b-48b6-a3c5-56bd6b88792a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bbe23a7b451494e954fd47eab70b644'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:12.970882 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1052/2126] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:34:12 2022] DELETE /identity/v3/users/6bbe23a7b451494e954fd47eab70b644 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:13.021165 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1075/2127] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:34:13.028383 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5300e12-137b-48b6-a3c5-56bd6b88792a admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:13.031017 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-d5300e12-137b-48b6-a3c5-56bd6b88792a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:13.068722 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b6c4f-f659-4806-82f4-5598afee8c6d None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:13.070097 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-cc8b6c4f-f659-4806-82f4-5598afee8c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:13.136522 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5300e12-137b-48b6-a3c5-56bd6b88792a admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:13.144014 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691452b4-aaec-4efe-8d67-cbe029fdcb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:13.144896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691452b4-aaec-4efe-8d67-cbe029fdcb4a admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:13.145395 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691452b4-aaec-4efe-8d67-cbe029fdcb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:13.145928 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691452b4-aaec-4efe-8d67-cbe029fdcb4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:13.146815 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691452b4-aaec-4efe-8d67-cbe029fdcb4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:13.152202 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691452b4-aaec-4efe-8d67-cbe029fdcb4a admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:13.187798 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b6c4f-f659-4806-82f4-5598afee8c6d None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:13.194826 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1053/2128] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:34:13 2022] POST /identity/v3/projects => generated 385 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:34:13.196687 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3a64b-40b0-451c-94b3-c2fca19e8822 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:13.197210 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3a64b-40b0-451c-94b3-c2fca19e8822 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:13.197700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3a64b-40b0-451c-94b3-c2fca19e8822 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:13.199036 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56284ca-dc84-4bbb-a107-414a0170f501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:13.200174 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56284ca-dc84-4bbb-a107-414a0170f501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:13.200174 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56284ca-dc84-4bbb-a107-414a0170f501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:13.200445 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56284ca-dc84-4bbb-a107-414a0170f501 admin admin] PATH_INFO: `/v3/projects/2cb3246131a9452b90c3a537d0be4143` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:13.200824 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1054/2129] 10.222.0.49 () {54 vars in 921 bytes} [Mon Sep 26 10:34:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:34:13.201710 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b3a64b-40b0-451c-94b3-c2fca19e8822 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:13.201843 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56284ca-dc84-4bbb-a107-414a0170f501 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cb3246131a9452b90c3a537d0be4143)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:13.204030 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56284ca-dc84-4bbb-a107-414a0170f501 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:13.276777 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:13.285155 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c56284ca-dc84-4bbb-a107-414a0170f501 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cb3246131a9452b90c3a537d0be4143', 'request_id': 'req-c56284ca-dc84-4bbb-a107-414a0170f501'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:13.293112 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c56284ca-dc84-4bbb-a107-414a0170f501 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cb3246131a9452b90c3a537d0be4143', 'request_id': 'req-c56284ca-dc84-4bbb-a107-414a0170f501'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:13.302432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c56284ca-dc84-4bbb-a107-414a0170f501 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cb3246131a9452b90c3a537d0be4143', 'request_id': 'req-c56284ca-dc84-4bbb-a107-414a0170f501'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:13.308721 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b3a64b-40b0-451c-94b3-c2fca19e8822 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:13.317156 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c56284ca-dc84-4bbb-a107-414a0170f501 admin admin] The token cache is being invalidate because project 2cb3246131a9452b90c3a537d0be4143 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:34:13.317701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c56284ca-dc84-4bbb-a107-414a0170f501 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:13.318507 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c56284ca-dc84-4bbb-a107-414a0170f501 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:13.338208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1076/2130] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:34:13 2022] DELETE /identity/v3/projects/2cb3246131a9452b90c3a537d0be4143 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:34:13.338208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba19362-1750-4973-bb68-89229ab4c85e None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:13.338208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba19362-1750-4973-bb68-89229ab4c85e None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:13.338208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba19362-1750-4973-bb68-89229ab4c85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:13.354794 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-3ba19362-1750-4973-bb68-89229ab4c85e None None] MFA Rules not processed for user `84b02e7351914cabb67462454b328d05`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:13.356250 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-3ba19362-1750-4973-bb68-89229ab4c85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:13.395492 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:13.430171 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7471caaf-7a68-4585-b9ad-a7cd6cf3afc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:13.431417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7471caaf-7a68-4585-b9ad-a7cd6cf3afc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:13.431961 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7471caaf-7a68-4585-b9ad-a7cd6cf3afc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:13.432336 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7471caaf-7a68-4585-b9ad-a7cd6cf3afc8 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:13.433088 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7471caaf-7a68-4585-b9ad-a7cd6cf3afc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:13.435309 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7471caaf-7a68-4585-b9ad-a7cd6cf3afc8 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:13.472605 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1055/2131] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:34:13 2022] POST /identity/v3/users => generated 299 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:34:13.477903 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7471caaf-7a68-4585-b9ad-a7cd6cf3afc8 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:13.479877 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-7471caaf-7a68-4585-b9ad-a7cd6cf3afc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:13.486631 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1077/2132] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Sep 26 10:34:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:34:13.487612 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba19362-1750-4973-bb68-89229ab4c85e None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:13.490073 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-3ba19362-1750-4973-bb68-89229ab4c85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:13.499430 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7471caaf-7a68-4585-b9ad-a7cd6cf3afc8 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:13.506711 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe71ec4-7afb-42fa-99fd-e9736741f29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:13.507597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe71ec4-7afb-42fa-99fd-e9736741f29f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:13.508189 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe71ec4-7afb-42fa-99fd-e9736741f29f admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:13.508489 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe71ec4-7afb-42fa-99fd-e9736741f29f admin admin] PATH_INFO: `/v3/users/398f3e872ad9499fbb26cc48ca9544d8` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:13.511050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe71ec4-7afb-42fa-99fd-e9736741f29f admin admin] RBAC: Authorizing `identity:delete_user(user_id=398f3e872ad9499fbb26cc48ca9544d8)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:13.513331 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe71ec4-7afb-42fa-99fd-e9736741f29f admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:13.520962 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba19362-1750-4973-bb68-89229ab4c85e None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:13.527220 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcdb4b7-5b94-4d86-8c97-7786878be4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:13.527953 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcdb4b7-5b94-4d86-8c97-7786878be4a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:13.528353 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcdb4b7-5b94-4d86-8c97-7786878be4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:13.528700 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcdb4b7-5b94-4d86-8c97-7786878be4a0 admin admin] PATH_INFO: `/v3/projects/69acb26f858847b58671f6ab36ee68e4` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:13.529921 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcdb4b7-5b94-4d86-8c97-7786878be4a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69acb26f858847b58671f6ab36ee68e4)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:13.532193 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcdb4b7-5b94-4d86-8c97-7786878be4a0 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:13.580972 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-3fe71ec4-7afb-42fa-99fd-e9736741f29f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '398f3e872ad9499fbb26cc48ca9544d8'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:13.581633 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-3fe71ec4-7afb-42fa-99fd-e9736741f29f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '398f3e872ad9499fbb26cc48ca9544d8'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:13.582454 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-3fe71ec4-7afb-42fa-99fd-e9736741f29f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '398f3e872ad9499fbb26cc48ca9544d8'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:13.590215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-cbcdb4b7-5b94-4d86-8c97-7786878be4a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69acb26f858847b58671f6ab36ee68e4', 'request_id': 'req-cbcdb4b7-5b94-4d86-8c97-7786878be4a0'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:13.593754 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-3fe71ec4-7afb-42fa-99fd-e9736741f29f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '398f3e872ad9499fbb26cc48ca9544d8'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:13.605341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-cbcdb4b7-5b94-4d86-8c97-7786878be4a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69acb26f858847b58671f6ab36ee68e4', 'request_id': 'req-cbcdb4b7-5b94-4d86-8c97-7786878be4a0'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:13.613476 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-3fe71ec4-7afb-42fa-99fd-e9736741f29f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '398f3e872ad9499fbb26cc48ca9544d8'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:13.616391 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-cbcdb4b7-5b94-4d86-8c97-7786878be4a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69acb26f858847b58671f6ab36ee68e4', 'request_id': 'req-cbcdb4b7-5b94-4d86-8c97-7786878be4a0'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:13.628025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-cbcdb4b7-5b94-4d86-8c97-7786878be4a0 admin admin] The token cache is being invalidate because project 69acb26f858847b58671f6ab36ee68e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:34:13.628318 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-cbcdb4b7-5b94-4d86-8c97-7786878be4a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:13.628423 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1056/2133] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:34:13 2022] DELETE /identity/v3/users/398f3e872ad9499fbb26cc48ca9544d8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:13.629171 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe71ec4-7afb-42fa-99fd-e9736741f29f admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:13.630513 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-3fe71ec4-7afb-42fa-99fd-e9736741f29f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:13.630819 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-cbcdb4b7-5b94-4d86-8c97-7786878be4a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:13.636084 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1078/2134] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:34:13 2022] DELETE /identity/v3/projects/69acb26f858847b58671f6ab36ee68e4 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:13.636950 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcdb4b7-5b94-4d86-8c97-7786878be4a0 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:13.638732 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-cbcdb4b7-5b94-4d86-8c97-7786878be4a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:13.691469 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:13.719617 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe71ec4-7afb-42fa-99fd-e9736741f29f admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:13.726642 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e23c19-32d0-4b4a-a6b2-584bd8a9802f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:13.727260 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e23c19-32d0-4b4a-a6b2-584bd8a9802f admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:13.727538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e23c19-32d0-4b4a-a6b2-584bd8a9802f admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:13.727811 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e23c19-32d0-4b4a-a6b2-584bd8a9802f admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:13.728457 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e23c19-32d0-4b4a-a6b2-584bd8a9802f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:13.730272 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e23c19-32d0-4b4a-a6b2-584bd8a9802f admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:13.737059 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcdb4b7-5b94-4d86-8c97-7786878be4a0 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:13.742697 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1057/2135] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:34:13 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:34:13.743574 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e23c19-32d0-4b4a-a6b2-584bd8a9802f admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:13.749913 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-28e23c19-32d0-4b4a-a6b2-584bd8a9802f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:13.764384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e23c19-32d0-4b4a-a6b2-584bd8a9802f admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:13.769374 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13c332-071d-4519-8fe1-ece35fb6a417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:13.770009 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13c332-071d-4519-8fe1-ece35fb6a417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:13.770557 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13c332-071d-4519-8fe1-ece35fb6a417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:13.770831 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13c332-071d-4519-8fe1-ece35fb6a417 admin admin] PATH_INFO: `/v3/users/38c21b9d4b7945d8bf6adab8eb7005b2` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:13.771990 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa13c332-071d-4519-8fe1-ece35fb6a417 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38c21b9d4b7945d8bf6adab8eb7005b2)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:13.774215 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa13c332-071d-4519-8fe1-ece35fb6a417 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:13.789605 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:13.825899 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1702ad-c475-4b2d-874d-16c996b2573c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84b02e7351914cabb67462454b328d05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:13.826538 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1702ad-c475-4b2d-874d-16c996b2573c service placement] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:13.826821 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1702ad-c475-4b2d-874d-16c996b2573c service placement] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:13.827093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1702ad-c475-4b2d-874d-16c996b2573c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:13.829396 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-fa1702ad-c475-4b2d-874d-16c996b2573c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:13.839009 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-fa13c332-071d-4519-8fe1-ece35fb6a417 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38c21b9d4b7945d8bf6adab8eb7005b2'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:13.839881 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-fa13c332-071d-4519-8fe1-ece35fb6a417 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38c21b9d4b7945d8bf6adab8eb7005b2'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:13.847799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-fa13c332-071d-4519-8fe1-ece35fb6a417 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38c21b9d4b7945d8bf6adab8eb7005b2'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:13.848797 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1702ad-c475-4b2d-874d-16c996b2573c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:13.850761 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1702ad-c475-4b2d-874d-16c996b2573c service placement] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:13.857545 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-fa13c332-071d-4519-8fe1-ece35fb6a417 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38c21b9d4b7945d8bf6adab8eb7005b2'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:13.864137 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1079/2136] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:34:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:34:13.866067 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1702ad-c475-4b2d-874d-16c996b2573c service placement] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:13.867609 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-fa1702ad-c475-4b2d-874d-16c996b2573c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:13.874074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-fa13c332-071d-4519-8fe1-ece35fb6a417 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38c21b9d4b7945d8bf6adab8eb7005b2'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:13.888400 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1702ad-c475-4b2d-874d-16c996b2573c service placement] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:13.893176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a8c40-83db-42be-926b-051d7b9df8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:13.893808 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a8c40-83db-42be-926b-051d7b9df8c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:13.895107 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a8c40-83db-42be-926b-051d7b9df8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:13.895107 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a8c40-83db-42be-926b-051d7b9df8c2 admin admin] PATH_INFO: `/v3/projects/76db848868cb417781b7bf896debf42c/users/2e6dc36b3c2540708e6b210d558ba220/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:13.902170 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1058/2137] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:34:13 2022] DELETE /identity/v3/users/38c21b9d4b7945d8bf6adab8eb7005b2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:13.916116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a8c40-83db-42be-926b-051d7b9df8c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76db848868cb417781b7bf896debf42c, user_id=2e6dc36b3c2540708e6b210d558ba220, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:13.919049 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a8c40-83db-42be-926b-051d7b9df8c2 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:13.935597 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1080/2138] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:34:13 2022] PUT /identity/v3/projects/76db848868cb417781b7bf896debf42c/users/2e6dc36b3c2540708e6b210d558ba220/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:13.939530 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13c332-071d-4519-8fe1-ece35fb6a417 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:13.941140 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-fa13c332-071d-4519-8fe1-ece35fb6a417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:13.994114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a8c40-83db-42be-926b-051d7b9df8c2 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:13.994114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-7e6a8c40-83db-42be-926b-051d7b9df8c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:14.049778 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13c332-071d-4519-8fe1-ece35fb6a417 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:14.100441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a8c40-83db-42be-926b-051d7b9df8c2 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:14.105450 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1c159-5f9c-49f1-a316-4d07e8895711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:14.106122 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1c159-5f9c-49f1-a316-4d07e8895711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:14.106999 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1c159-5f9c-49f1-a316-4d07e8895711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:14.107283 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1c159-5f9c-49f1-a316-4d07e8895711 admin admin] PATH_INFO: `/v3/projects/c64a07f1fc58419e8dbd23d28a43e98b` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:14.108637 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a1c159-5f9c-49f1-a316-4d07e8895711 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c64a07f1fc58419e8dbd23d28a43e98b)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:14.111025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a1c159-5f9c-49f1-a316-4d07e8895711 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:14.133846 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a972b-150c-491a-8c31-0e366ba81a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:14.134495 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a972b-150c-491a-8c31-0e366ba81a3b admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:14.134783 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a972b-150c-491a-8c31-0e366ba81a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:14.135058 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a972b-150c-491a-8c31-0e366ba81a3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:14.136897 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753a972b-150c-491a-8c31-0e366ba81a3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:14.138620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753a972b-150c-491a-8c31-0e366ba81a3b admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:14.140318 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:14.153784 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1059/2139] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:34:13 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:34:14.157538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a972b-150c-491a-8c31-0e366ba81a3b admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:14.174692 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a972b-150c-491a-8c31-0e366ba81a3b admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:14.182263 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-e2a1c159-5f9c-49f1-a316-4d07e8895711 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c64a07f1fc58419e8dbd23d28a43e98b', 'request_id': 'req-e2a1c159-5f9c-49f1-a316-4d07e8895711'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:14.189611 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-e2a1c159-5f9c-49f1-a316-4d07e8895711 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c64a07f1fc58419e8dbd23d28a43e98b', 'request_id': 'req-e2a1c159-5f9c-49f1-a316-4d07e8895711'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:14.198849 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-e2a1c159-5f9c-49f1-a316-4d07e8895711 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c64a07f1fc58419e8dbd23d28a43e98b', 'request_id': 'req-e2a1c159-5f9c-49f1-a316-4d07e8895711'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:14.210440 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-e2a1c159-5f9c-49f1-a316-4d07e8895711 admin admin] The token cache is being invalidate because project c64a07f1fc58419e8dbd23d28a43e98b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:34:14.211068 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-e2a1c159-5f9c-49f1-a316-4d07e8895711 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:14.211585 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-e2a1c159-5f9c-49f1-a316-4d07e8895711 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:14.213569 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1081/2140] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:34:13 2022] DELETE /identity/v3/projects/c64a07f1fc58419e8dbd23d28a43e98b => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:14.218439 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ecf06-5fff-4304-9ac5-b380100d6b0b None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:14.218734 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ecf06-5fff-4304-9ac5-b380100d6b0b None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:14.219013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ecf06-5fff-4304-9ac5-b380100d6b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:14.240952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-e94ecf06-5fff-4304-9ac5-b380100d6b0b None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:14.242279 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-e94ecf06-5fff-4304-9ac5-b380100d6b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:14.254503 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc0034-a923-4254-9289-bc3f2f8af794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:14.256031 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc0034-a923-4254-9289-bc3f2f8af794 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:14.256491 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc0034-a923-4254-9289-bc3f2f8af794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:14.256935 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc0034-a923-4254-9289-bc3f2f8af794 admin admin] PATH_INFO: `/v3/projects/76db848868cb417781b7bf896debf42c/users/2e6dc36b3c2540708e6b210d558ba220/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:14.259443 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1082/2141] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:34:14.259877 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc0034-a923-4254-9289-bc3f2f8af794 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76db848868cb417781b7bf896debf42c, user_id=2e6dc36b3c2540708e6b210d558ba220, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:14.263153 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc0034-a923-4254-9289-bc3f2f8af794 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:14.267663 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa16fb-ab9d-4677-9545-93e997685106 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:14.267947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa16fb-ab9d-4677-9545-93e997685106 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:14.268223 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa16fb-ab9d-4677-9545-93e997685106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:14.274304 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1060/2142] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:34:14 2022] PUT /identity/v3/projects/76db848868cb417781b7bf896debf42c/users/2e6dc36b3c2540708e6b210d558ba220/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:14.275111 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc0034-a923-4254-9289-bc3f2f8af794 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:14.276708 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-88bc0034-a923-4254-9289-bc3f2f8af794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:14.310619 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-a0fa16fb-ab9d-4677-9545-93e997685106 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:14.311837 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-a0fa16fb-ab9d-4677-9545-93e997685106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:14.344778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:14.349804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc0034-a923-4254-9289-bc3f2f8af794 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:14.355960 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a1dd98-db69-4ec1-9190-3efdb7ce7646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:14.357514 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1dd98-db69-4ec1-9190-3efdb7ce7646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:14.357799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1dd98-db69-4ec1-9190-3efdb7ce7646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:14.358099 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1dd98-db69-4ec1-9190-3efdb7ce7646 admin admin] PATH_INFO: `/v3/projects/e3a5c3a487074201b356c268b95c2a5a` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:14.359281 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a1dd98-db69-4ec1-9190-3efdb7ce7646 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3a5c3a487074201b356c268b95c2a5a)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:14.361224 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a1dd98-db69-4ec1-9190-3efdb7ce7646 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:14.379074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1083/2143] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:34:14.380337 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f961d-f9ce-4117-babf-4c4a2e4388e5 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:14.380624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f961d-f9ce-4117-babf-4c4a2e4388e5 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:14.380898 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f961d-f9ce-4117-babf-4c4a2e4388e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:14.410681 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-66a1dd98-db69-4ec1-9190-3efdb7ce7646 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3a5c3a487074201b356c268b95c2a5a', 'request_id': 'req-66a1dd98-db69-4ec1-9190-3efdb7ce7646'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:14.414243 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-6f7f961d-f9ce-4117-babf-4c4a2e4388e5 None None] MFA Rules not processed for user `2e6dc36b3c2540708e6b210d558ba220`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:14.415509 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-6f7f961d-f9ce-4117-babf-4c4a2e4388e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:14.417544 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-66a1dd98-db69-4ec1-9190-3efdb7ce7646 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3a5c3a487074201b356c268b95c2a5a', 'request_id': 'req-66a1dd98-db69-4ec1-9190-3efdb7ce7646'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:14.423073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-66a1dd98-db69-4ec1-9190-3efdb7ce7646 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3a5c3a487074201b356c268b95c2a5a', 'request_id': 'req-66a1dd98-db69-4ec1-9190-3efdb7ce7646'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:14.436518 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-66a1dd98-db69-4ec1-9190-3efdb7ce7646 admin admin] The token cache is being invalidate because project e3a5c3a487074201b356c268b95c2a5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:34:14.436837 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-66a1dd98-db69-4ec1-9190-3efdb7ce7646 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:14.437469 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-66a1dd98-db69-4ec1-9190-3efdb7ce7646 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:14.439375 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1061/2144] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:34:14 2022] DELETE /identity/v3/projects/e3a5c3a487074201b356c268b95c2a5a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:14.439914 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a1dd98-db69-4ec1-9190-3efdb7ce7646 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:14.441284 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-66a1dd98-db69-4ec1-9190-3efdb7ce7646 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:14.468582 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:14.491508 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1084/2145] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:14 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:34:14.498361 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87246b-b3b4-4dcf-9300-f37520a6efd6 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:14.498571 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87246b-b3b4-4dcf-9300-f37520a6efd6 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:14.498851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87246b-b3b4-4dcf-9300-f37520a6efd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:14.517523 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a1dd98-db69-4ec1-9190-3efdb7ce7646 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:14.520067 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-6a87246b-b3b4-4dcf-9300-f37520a6efd6 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:14.521423 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-6a87246b-b3b4-4dcf-9300-f37520a6efd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:14.522579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898af698-22b0-49b8-a391-9c2b1e775dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:14.527371 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898af698-22b0-49b8-a391-9c2b1e775dac admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:14.527779 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898af698-22b0-49b8-a391-9c2b1e775dac admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:14.528149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898af698-22b0-49b8-a391-9c2b1e775dac admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:14.528919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898af698-22b0-49b8-a391-9c2b1e775dac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:34:14.529304 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898af698-22b0-49b8-a391-9c2b1e775dac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:14.531199 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898af698-22b0-49b8-a391-9c2b1e775dac admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:14.538214 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1085/2146] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:34:14.539441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87246b-b3b4-4dcf-9300-f37520a6efd6 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:14.541050 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-6a87246b-b3b4-4dcf-9300-f37520a6efd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:14.547349 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1062/2147] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:34:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:34:14.547349 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36eafa-be33-42e8-81fa-a136e416741c None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:14.547349 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36eafa-be33-42e8-81fa-a136e416741c None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:14.547349 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba36eafa-be33-42e8-81fa-a136e416741c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:14.567158 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-ba36eafa-be33-42e8-81fa-a136e416741c None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:14.568378 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-ba36eafa-be33-42e8-81fa-a136e416741c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:14.577388 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1063/2148] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:34:14.578389 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36eafa-be33-42e8-81fa-a136e416741c None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:14.589310 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba36eafa-be33-42e8-81fa-a136e416741c None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:14.594053 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453398c-de89-42b7-971d-0c3673fd9dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:14.594651 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453398c-de89-42b7-971d-0c3673fd9dbc admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:14.594929 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453398c-de89-42b7-971d-0c3673fd9dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:14.595210 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453398c-de89-42b7-971d-0c3673fd9dbc admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:14.595832 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4453398c-de89-42b7-971d-0c3673fd9dbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:14.597728 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4453398c-de89-42b7-971d-0c3673fd9dbc admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:14.609679 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87246b-b3b4-4dcf-9300-f37520a6efd6 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:14.614582 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a59216-65ca-4be7-b6d2-e449ed253f3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:14.615200 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a59216-65ca-4be7-b6d2-e449ed253f3a service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:14.615479 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a59216-65ca-4be7-b6d2-e449ed253f3a service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:14.615752 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a59216-65ca-4be7-b6d2-e449ed253f3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:14.617716 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-e9a59216-65ca-4be7-b6d2-e449ed253f3a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:14.628724 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1064/2149] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:34:14 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:34:14.629709 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453398c-de89-42b7-971d-0c3673fd9dbc admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:14.679300 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a59216-65ca-4be7-b6d2-e449ed253f3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:14.680722 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a59216-65ca-4be7-b6d2-e449ed253f3a service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:14.691414 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1086/2150] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:34:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:34:14.692263 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a59216-65ca-4be7-b6d2-e449ed253f3a service cinder] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:14.696103 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453398c-de89-42b7-971d-0c3673fd9dbc admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:14.701381 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f86558-97a2-45d3-bb0d-ba08e7cecdf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:14.702129 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f86558-97a2-45d3-bb0d-ba08e7cecdf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:14.702507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f86558-97a2-45d3-bb0d-ba08e7cecdf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:14.702880 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f86558-97a2-45d3-bb0d-ba08e7cecdf5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:14.703637 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f86558-97a2-45d3-bb0d-ba08e7cecdf5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:34:14.703747 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a59216-65ca-4be7-b6d2-e449ed253f3a service cinder] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:14.704106 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f86558-97a2-45d3-bb0d-ba08e7cecdf5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:14.705828 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f86558-97a2-45d3-bb0d-ba08e7cecdf5 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:14.708426 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef6c80-0317-4e63-b669-dd816b243565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:14.709024 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef6c80-0317-4e63-b669-dd816b243565 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:14.709291 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef6c80-0317-4e63-b669-dd816b243565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:14.709568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef6c80-0317-4e63-b669-dd816b243565 admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:14.710220 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef6c80-0317-4e63-b669-dd816b243565 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:14.712099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef6c80-0317-4e63-b669-dd816b243565 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:14.720149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1065/2151] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:34:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:34:14.721426 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:14.724906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f86558-97a2-45d3-bb0d-ba08e7cecdf5 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:14.737063 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f86558-97a2-45d3-bb0d-ba08e7cecdf5 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:14.741079 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1087/2152] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:34:14 2022] POST /identity/v3/users => generated 295 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:34:14.745289 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef6c80-0317-4e63-b669-dd816b243565 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:14.746552 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b8146-6a17-4208-91ab-56f46859849d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:14.747243 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b8146-6a17-4208-91ab-56f46859849d admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:14.747628 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b8146-6a17-4208-91ab-56f46859849d admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:14.747966 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b8146-6a17-4208-91ab-56f46859849d admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:14.748661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b8146-6a17-4208-91ab-56f46859849d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:14.750839 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b8146-6a17-4208-91ab-56f46859849d admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:14.755540 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef6c80-0317-4e63-b669-dd816b243565 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:14.760818 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d993efb-f307-4a06-b8da-eda0df42c4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:14.761548 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d993efb-f307-4a06-b8da-eda0df42c4cc admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:14.761899 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d993efb-f307-4a06-b8da-eda0df42c4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:14.762269 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d993efb-f307-4a06-b8da-eda0df42c4cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:14.763160 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d993efb-f307-4a06-b8da-eda0df42c4cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:14.764835 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d993efb-f307-4a06-b8da-eda0df42c4cc admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:14.778547 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1088/2153] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:34:14.782453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d993efb-f307-4a06-b8da-eda0df42c4cc admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:14.794442 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1066/2154] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:34:14 2022] POST /identity/v3/projects => generated 369 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:34:14.794442 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b8146-6a17-4208-91ab-56f46859849d admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:14.796063 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d993efb-f307-4a06-b8da-eda0df42c4cc admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:14.861345 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5e272-aabf-4a44-b252-305f2f945f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:14.862254 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5e272-aabf-4a44-b252-305f2f945f58 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:14.862534 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5e272-aabf-4a44-b252-305f2f945f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:14.862806 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5e272-aabf-4a44-b252-305f2f945f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:14.863453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a5e272-aabf-4a44-b252-305f2f945f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:14.865082 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a5e272-aabf-4a44-b252-305f2f945f58 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:14.872126 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:14.876740 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b8146-6a17-4208-91ab-56f46859849d admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:14.881432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1089/2155] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:34:14.882642 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5e272-aabf-4a44-b252-305f2f945f58 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:14.882761 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b53615-f1b5-4d70-b1a1-76db15508fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:14.883264 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b53615-f1b5-4d70-b1a1-76db15508fe2 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:14.883549 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b53615-f1b5-4d70-b1a1-76db15508fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:14.884368 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b53615-f1b5-4d70-b1a1-76db15508fe2 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:14.885820 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b53615-f1b5-4d70-b1a1-76db15508fe2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:14.888513 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b53615-f1b5-4d70-b1a1-76db15508fe2 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:14.893952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5e272-aabf-4a44-b252-305f2f945f58 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:14.898858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef6203-c591-4c59-996f-402f623a78f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:14.899688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef6203-c591-4c59-996f-402f623a78f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:14.899967 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef6203-c591-4c59-996f-402f623a78f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:14.900247 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef6203-c591-4c59-996f-402f623a78f2 admin admin] PATH_INFO: `/v3/projects/38cccae871b94a9db82b5017c2c497c1/users/8c8ebfdad6ab437eb7dda111fee84205/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:14.914789 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1067/2156] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:34:14 2022] POST /identity/v3/users => generated 285 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:34:14.919042 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b53615-f1b5-4d70-b1a1-76db15508fe2 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:14.920901 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ef6203-c591-4c59-996f-402f623a78f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38cccae871b94a9db82b5017c2c497c1, user_id=8c8ebfdad6ab437eb7dda111fee84205, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:14.923937 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ef6203-c591-4c59-996f-402f623a78f2 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:14.930535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b53615-f1b5-4d70-b1a1-76db15508fe2 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:14.935731 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af27c42-3722-4d99-a822-c3d1afbcc62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:14.936343 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af27c42-3722-4d99-a822-c3d1afbcc62a admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:14.936625 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af27c42-3722-4d99-a822-c3d1afbcc62a admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:14.936901 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af27c42-3722-4d99-a822-c3d1afbcc62a admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:14.937627 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af27c42-3722-4d99-a822-c3d1afbcc62a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:14.939257 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af27c42-3722-4d99-a822-c3d1afbcc62a admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:14.942289 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1090/2157] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:34:14 2022] PUT /identity/v3/projects/38cccae871b94a9db82b5017c2c497c1/users/8c8ebfdad6ab437eb7dda111fee84205/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:34:14.947052 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6e672-6089-451f-bcfc-059691dc8e96 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:14.947395 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6e672-6089-451f-bcfc-059691dc8e96 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:14.947726 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6e672-6089-451f-bcfc-059691dc8e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:14.955130 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1068/2158] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:34:14.957106 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af27c42-3722-4d99-a822-c3d1afbcc62a admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:14.969872 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af27c42-3722-4d99-a822-c3d1afbcc62a admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:14.979616 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-26a6e672-6089-451f-bcfc-059691dc8e96 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:14.980836 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-26a6e672-6089-451f-bcfc-059691dc8e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:15.076341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1091/2159] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:34:15.091604 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a6e672-6089-451f-bcfc-059691dc8e96 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:15.095004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-26a6e672-6089-451f-bcfc-059691dc8e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:15.122474 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518417ce-8adc-4d7e-970d-dcef7b201d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:15.123291 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518417ce-8adc-4d7e-970d-dcef7b201d87 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:15.123564 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518417ce-8adc-4d7e-970d-dcef7b201d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:15.123833 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518417ce-8adc-4d7e-970d-dcef7b201d87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:15.124476 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518417ce-8adc-4d7e-970d-dcef7b201d87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:15.126150 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518417ce-8adc-4d7e-970d-dcef7b201d87 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:15.142227 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1069/2160] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:34:15.143345 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518417ce-8adc-4d7e-970d-dcef7b201d87 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:15.156586 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518417ce-8adc-4d7e-970d-dcef7b201d87 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:15.161027 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a6e672-6089-451f-bcfc-059691dc8e96 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:15.161297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84002d-fba4-4fc6-a198-05a465deb710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:15.161951 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84002d-fba4-4fc6-a198-05a465deb710 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:15.162277 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84002d-fba4-4fc6-a198-05a465deb710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:15.162702 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84002d-fba4-4fc6-a198-05a465deb710 admin admin] PATH_INFO: `/v3/projects/6c0c5298b56944e1a342ee54ecdf3ceb/users/ab41daa8d6624064ae4e4a8009905667/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:15.165081 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732747d-e830-49b9-8b10-3d6c1f4a7e47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:15.165692 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732747d-e830-49b9-8b10-3d6c1f4a7e47 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:15.165976 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732747d-e830-49b9-8b10-3d6c1f4a7e47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:15.166297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732747d-e830-49b9-8b10-3d6c1f4a7e47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:15.172458 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:15.178412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732747d-e830-49b9-8b10-3d6c1f4a7e47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:15.180736 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732747d-e830-49b9-8b10-3d6c1f4a7e47 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:15.181672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c84002d-fba4-4fc6-a198-05a465deb710 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c0c5298b56944e1a342ee54ecdf3ceb, user_id=ab41daa8d6624064ae4e4a8009905667, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:15.184687 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c84002d-fba4-4fc6-a198-05a465deb710 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:15.192512 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1092/2161] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:34:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:34:15.200039 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1070/2162] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:34:15 2022] PUT /identity/v3/projects/6c0c5298b56944e1a342ee54ecdf3ceb/users/ab41daa8d6624064ae4e4a8009905667/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:15.210114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14384853-d3bd-4126-8d49-cd426718c62b None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:15.210406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14384853-d3bd-4126-8d49-cd426718c62b None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:15.210679 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14384853-d3bd-4126-8d49-cd426718c62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:15.241492 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-14384853-d3bd-4126-8d49-cd426718c62b None None] MFA Rules not processed for user `ab41daa8d6624064ae4e4a8009905667`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:15.242841 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-14384853-d3bd-4126-8d49-cd426718c62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:15.322872 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1093/2163] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:34:15.335959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84002d-fba4-4fc6-a198-05a465deb710 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:15.338186 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-2c84002d-fba4-4fc6-a198-05a465deb710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:15.413331 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84002d-fba4-4fc6-a198-05a465deb710 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:15.417824 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7722a83-d295-4485-a9f5-8f5ff830b588 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41b242fbe2504f8cac26ce4ff46f5390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:15.418716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7722a83-d295-4485-a9f5-8f5ff830b588 service glance] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:15.419067 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7722a83-d295-4485-a9f5-8f5ff830b588 service glance] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:15.419419 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7722a83-d295-4485-a9f5-8f5ff830b588 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:15.433340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7722a83-d295-4485-a9f5-8f5ff830b588 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:15.434894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7722a83-d295-4485-a9f5-8f5ff830b588 service glance] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:15.444541 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1071/2164] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:34:15 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:34:16.409442 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14384853-d3bd-4126-8d49-cd426718c62b None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:16.501841 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14384853-d3bd-4126-8d49-cd426718c62b None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:16.506868 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164af9ff-b0ab-4476-9305-d0a3e416aa6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:16.507956 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164af9ff-b0ab-4476-9305-d0a3e416aa6e service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:16.508087 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164af9ff-b0ab-4476-9305-d0a3e416aa6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:16.508421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164af9ff-b0ab-4476-9305-d0a3e416aa6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:16.510273 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-164af9ff-b0ab-4476-9305-d0a3e416aa6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:16.605861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164af9ff-b0ab-4476-9305-d0a3e416aa6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:16.607348 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164af9ff-b0ab-4476-9305-d0a3e416aa6e service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:16.619070 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1094/2165] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:34:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:34:17.318593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7722a83-d295-4485-a9f5-8f5ff830b588 service glance] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:17.330985 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7722a83-d295-4485-a9f5-8f5ff830b588 service glance] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:17.336436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d42fbd-47f8-4f09-9f49-2d29ecf48d85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:17.337056 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d42fbd-47f8-4f09-9f49-2d29ecf48d85 service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:17.337751 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d42fbd-47f8-4f09-9f49-2d29ecf48d85 service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:17.337751 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d42fbd-47f8-4f09-9f49-2d29ecf48d85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:17.394893 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:17.409937 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d42fbd-47f8-4f09-9f49-2d29ecf48d85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:17.411404 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d42fbd-47f8-4f09-9f49-2d29ecf48d85 service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:17.420984 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1072/2166] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:34:17 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:34:19.626896 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e25af-80b7-4945-8aa6-4dd58c49d590 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:19.627421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e25af-80b7-4945-8aa6-4dd58c49d590 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:19.627547 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e25af-80b7-4945-8aa6-4dd58c49d590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:19.665629 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-da2e25af-80b7-4945-8aa6-4dd58c49d590 None None] MFA Rules not processed for user `8c8ebfdad6ab437eb7dda111fee84205`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:19.666935 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-da2e25af-80b7-4945-8aa6-4dd58c49d590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:19.694417 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:19.742911 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1095/2167] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:19 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:34:19.756633 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d42fbd-47f8-4f09-9f49-2d29ecf48d85 service cinder] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:19.758203 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-24d42fbd-47f8-4f09-9f49-2d29ecf48d85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:19.858950 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d42fbd-47f8-4f09-9f49-2d29ecf48d85 service cinder] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:19.864807 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a7f0df-bc9a-4d95-bd2e-2897521eb2d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:19.865427 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7f0df-bc9a-4d95-bd2e-2897521eb2d1 service nova] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:19.865705 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7f0df-bc9a-4d95-bd2e-2897521eb2d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:19.865979 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7f0df-bc9a-4d95-bd2e-2897521eb2d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:19.874983 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a7f0df-bc9a-4d95-bd2e-2897521eb2d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:19.876599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a7f0df-bc9a-4d95-bd2e-2897521eb2d1 service nova] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:19.886922 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1073/2168] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:34:19 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:34:21.478005 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4acb82-932b-4ea1-87e3-9bdc965c47df None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:21.478822 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4acb82-932b-4ea1-87e3-9bdc965c47df None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:21.479097 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4acb82-932b-4ea1-87e3-9bdc965c47df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:21.515811 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-6b4acb82-932b-4ea1-87e3-9bdc965c47df None None] MFA Rules not processed for user `8c8ebfdad6ab437eb7dda111fee84205`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:21.517386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-6b4acb82-932b-4ea1-87e3-9bdc965c47df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:21.527640 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1096/2169] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:21 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:34:21.730314 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a7f0df-bc9a-4d95-bd2e-2897521eb2d1 service nova] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:21.733728 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-47a7f0df-bc9a-4d95-bd2e-2897521eb2d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:21.838798 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a7f0df-bc9a-4d95-bd2e-2897521eb2d1 service nova] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:21.849113 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec26943-3375-4ff7-9a56-b11b412b9c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:21.850827 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec26943-3375-4ff7-9a56-b11b412b9c8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:21.851270 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec26943-3375-4ff7-9a56-b11b412b9c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:21.851627 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec26943-3375-4ff7-9a56-b11b412b9c8e admin admin] PATH_INFO: `/v3/users/a8f7651292da480496db678f4269a249` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:21.852954 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec26943-3375-4ff7-9a56-b11b412b9c8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8f7651292da480496db678f4269a249)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:21.855813 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec26943-3375-4ff7-9a56-b11b412b9c8e admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:21.920947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-bec26943-3375-4ff7-9a56-b11b412b9c8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8f7651292da480496db678f4269a249'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:21.921766 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-bec26943-3375-4ff7-9a56-b11b412b9c8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8f7651292da480496db678f4269a249'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:21.922786 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-bec26943-3375-4ff7-9a56-b11b412b9c8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8f7651292da480496db678f4269a249'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:21.930393 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-bec26943-3375-4ff7-9a56-b11b412b9c8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8f7651292da480496db678f4269a249'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:21.947881 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:21.951635 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-bec26943-3375-4ff7-9a56-b11b412b9c8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8f7651292da480496db678f4269a249'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:21.967887 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1074/2170] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:34:21 2022] DELETE /identity/v3/users/a8f7651292da480496db678f4269a249 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:21.973205 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443bd18-a699-4686-9aab-81ab2ef7aad3 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:21.973760 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443bd18-a699-4686-9aab-81ab2ef7aad3 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:21.974276 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443bd18-a699-4686-9aab-81ab2ef7aad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:21.999789 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-a443bd18-a699-4686-9aab-81ab2ef7aad3 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:22.001160 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-a443bd18-a699-4686-9aab-81ab2ef7aad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:22.073425 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1097/2171] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:34:22.083732 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec26943-3375-4ff7-9a56-b11b412b9c8e admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:22.085286 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-bec26943-3375-4ff7-9a56-b11b412b9c8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:22.160927 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec26943-3375-4ff7-9a56-b11b412b9c8e admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:22.163910 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e99bb-5ee1-462a-82e6-0ede3d58c374 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:22.164522 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e99bb-5ee1-462a-82e6-0ede3d58c374 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:22.164799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e99bb-5ee1-462a-82e6-0ede3d58c374 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:22.165067 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e99bb-5ee1-462a-82e6-0ede3d58c374 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:22.176781 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0e99bb-5ee1-462a-82e6-0ede3d58c374 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:22.178391 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0e99bb-5ee1-462a-82e6-0ede3d58c374 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:22.192628 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1075/2172] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:34:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:34:22.538756 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443bd18-a699-4686-9aab-81ab2ef7aad3 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:22.540666 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-a443bd18-a699-4686-9aab-81ab2ef7aad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:22.562286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443bd18-a699-4686-9aab-81ab2ef7aad3 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:22.575155 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c260d2c7-9569-4767-a44a-58303ac3d306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:22.576205 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260d2c7-9569-4767-a44a-58303ac3d306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:22.576624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260d2c7-9569-4767-a44a-58303ac3d306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:22.577029 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260d2c7-9569-4767-a44a-58303ac3d306 admin admin] PATH_INFO: `/v3/projects/688e567df52e44519e1e79da7b8b4d4e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:22.578660 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c260d2c7-9569-4767-a44a-58303ac3d306 admin admin] RBAC: Authorizing `identity:delete_project(project_id=688e567df52e44519e1e79da7b8b4d4e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:22.580753 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c260d2c7-9569-4767-a44a-58303ac3d306 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:22.656401 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c260d2c7-9569-4767-a44a-58303ac3d306 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '688e567df52e44519e1e79da7b8b4d4e', 'request_id': 'req-c260d2c7-9569-4767-a44a-58303ac3d306'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:22.665866 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c260d2c7-9569-4767-a44a-58303ac3d306 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '688e567df52e44519e1e79da7b8b4d4e', 'request_id': 'req-c260d2c7-9569-4767-a44a-58303ac3d306'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:22.677091 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c260d2c7-9569-4767-a44a-58303ac3d306 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '688e567df52e44519e1e79da7b8b4d4e', 'request_id': 'req-c260d2c7-9569-4767-a44a-58303ac3d306'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:22.701146 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c260d2c7-9569-4767-a44a-58303ac3d306 admin admin] The token cache is being invalidate because project 688e567df52e44519e1e79da7b8b4d4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:34:22.701714 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c260d2c7-9569-4767-a44a-58303ac3d306 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:22.703036 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c260d2c7-9569-4767-a44a-58303ac3d306 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:22.716708 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1098/2173] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:34:22 2022] DELETE /identity/v3/projects/688e567df52e44519e1e79da7b8b4d4e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:22.721071 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e99bb-5ee1-462a-82e6-0ede3d58c374 service neutron] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:22.738660 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e99bb-5ee1-462a-82e6-0ede3d58c374 service neutron] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:22.856304 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab3e027-4e62-4c63-9174-d16197d36cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:22.857164 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3e027-4e62-4c63-9174-d16197d36cd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:22.857645 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3e027-4e62-4c63-9174-d16197d36cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:22.858233 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3e027-4e62-4c63-9174-d16197d36cd8 admin admin] PATH_INFO: `/v3/users/d013ac4b40e74366b26980fc7ec4369b` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:22.859722 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab3e027-4e62-4c63-9174-d16197d36cd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d013ac4b40e74366b26980fc7ec4369b)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:22.862236 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab3e027-4e62-4c63-9174-d16197d36cd8 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:22.938225 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-aab3e027-4e62-4c63-9174-d16197d36cd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd013ac4b40e74366b26980fc7ec4369b'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:22.939029 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-aab3e027-4e62-4c63-9174-d16197d36cd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd013ac4b40e74366b26980fc7ec4369b'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:22.940332 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-aab3e027-4e62-4c63-9174-d16197d36cd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd013ac4b40e74366b26980fc7ec4369b'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:22.949277 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-aab3e027-4e62-4c63-9174-d16197d36cd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd013ac4b40e74366b26980fc7ec4369b'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:22.966952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-aab3e027-4e62-4c63-9174-d16197d36cd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd013ac4b40e74366b26980fc7ec4369b'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:22.999167 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1076/2174] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:34:22 2022] DELETE /identity/v3/users/d013ac4b40e74366b26980fc7ec4369b => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:23.345144 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c260d2c7-9569-4767-a44a-58303ac3d306 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:23.346168 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-c260d2c7-9569-4767-a44a-58303ac3d306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:23.367528 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c260d2c7-9569-4767-a44a-58303ac3d306 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:23.423691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:23.450338 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadcb2a3-9c78-4f27-8b02-319869e96b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:23.458129 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadcb2a3-9c78-4f27-8b02-319869e96b1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:23.458911 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadcb2a3-9c78-4f27-8b02-319869e96b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:23.459341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadcb2a3-9c78-4f27-8b02-319869e96b1f admin admin] PATH_INFO: `/v3/projects/1644ce5f4f474a3e9ea5058ddc55ef14` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:23.460914 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadcb2a3-9c78-4f27-8b02-319869e96b1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1644ce5f4f474a3e9ea5058ddc55ef14)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:23.464731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadcb2a3-9c78-4f27-8b02-319869e96b1f admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:23.580662 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-fadcb2a3-9c78-4f27-8b02-319869e96b1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1644ce5f4f474a3e9ea5058ddc55ef14', 'request_id': 'req-fadcb2a3-9c78-4f27-8b02-319869e96b1f'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:23.589327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-fadcb2a3-9c78-4f27-8b02-319869e96b1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1644ce5f4f474a3e9ea5058ddc55ef14', 'request_id': 'req-fadcb2a3-9c78-4f27-8b02-319869e96b1f'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:23.601020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-fadcb2a3-9c78-4f27-8b02-319869e96b1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1644ce5f4f474a3e9ea5058ddc55ef14', 'request_id': 'req-fadcb2a3-9c78-4f27-8b02-319869e96b1f'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:23.614600 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-fadcb2a3-9c78-4f27-8b02-319869e96b1f admin admin] The token cache is being invalidate because project 1644ce5f4f474a3e9ea5058ddc55ef14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:34:23.614897 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-fadcb2a3-9c78-4f27-8b02-319869e96b1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:23.615546 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-fadcb2a3-9c78-4f27-8b02-319869e96b1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:23.620129 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1099/2175] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:34:23 2022] DELETE /identity/v3/projects/1644ce5f4f474a3e9ea5058ddc55ef14 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:23.621836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab3e027-4e62-4c63-9174-d16197d36cd8 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:23.623473 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-aab3e027-4e62-4c63-9174-d16197d36cd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:23.638601 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab3e027-4e62-4c63-9174-d16197d36cd8 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:23.647597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:23.713952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4523c4-f807-404e-9ac1-b352a6cd22dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:23.714619 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4523c4-f807-404e-9ac1-b352a6cd22dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:23.714897 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4523c4-f807-404e-9ac1-b352a6cd22dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:23.715950 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4523c4-f807-404e-9ac1-b352a6cd22dc admin admin] PATH_INFO: `/v3/users/2e6dc36b3c2540708e6b210d558ba220` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:23.717250 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4523c4-f807-404e-9ac1-b352a6cd22dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e6dc36b3c2540708e6b210d558ba220)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:23.719650 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4523c4-f807-404e-9ac1-b352a6cd22dc admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:23.784067 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-aa4523c4-f807-404e-9ac1-b352a6cd22dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e6dc36b3c2540708e6b210d558ba220'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:23.785833 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-aa4523c4-f807-404e-9ac1-b352a6cd22dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e6dc36b3c2540708e6b210d558ba220'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:23.786740 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-aa4523c4-f807-404e-9ac1-b352a6cd22dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e6dc36b3c2540708e6b210d558ba220'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:23.813646 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-aa4523c4-f807-404e-9ac1-b352a6cd22dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e6dc36b3c2540708e6b210d558ba220'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:23.832841 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-aa4523c4-f807-404e-9ac1-b352a6cd22dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e6dc36b3c2540708e6b210d558ba220'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:23.855651 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1077/2176] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:34:23 2022] DELETE /identity/v3/users/2e6dc36b3c2540708e6b210d558ba220 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:24.491924 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadcb2a3-9c78-4f27-8b02-319869e96b1f admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:24.493546 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-fadcb2a3-9c78-4f27-8b02-319869e96b1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:24.507959 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadcb2a3-9c78-4f27-8b02-319869e96b1f admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:24.663093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d48162-ff7d-4480-a25b-f87b7124698a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:24.665194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d48162-ff7d-4480-a25b-f87b7124698a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:24.666151 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d48162-ff7d-4480-a25b-f87b7124698a admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:24.668250 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d48162-ff7d-4480-a25b-f87b7124698a admin admin] PATH_INFO: `/v3/projects/76db848868cb417781b7bf896debf42c` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:24.669935 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d48162-ff7d-4480-a25b-f87b7124698a admin admin] RBAC: Authorizing `identity:delete_project(project_id=76db848868cb417781b7bf896debf42c)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:24.678069 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d48162-ff7d-4480-a25b-f87b7124698a admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:24.741020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-71d48162-ff7d-4480-a25b-f87b7124698a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76db848868cb417781b7bf896debf42c', 'request_id': 'req-71d48162-ff7d-4480-a25b-f87b7124698a'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:24.748585 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-71d48162-ff7d-4480-a25b-f87b7124698a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76db848868cb417781b7bf896debf42c', 'request_id': 'req-71d48162-ff7d-4480-a25b-f87b7124698a'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:24.757001 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-71d48162-ff7d-4480-a25b-f87b7124698a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76db848868cb417781b7bf896debf42c', 'request_id': 'req-71d48162-ff7d-4480-a25b-f87b7124698a'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:24.768369 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-71d48162-ff7d-4480-a25b-f87b7124698a admin admin] The token cache is being invalidate because project 76db848868cb417781b7bf896debf42c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:34:24.768663 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-71d48162-ff7d-4480-a25b-f87b7124698a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:24.769365 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-71d48162-ff7d-4480-a25b-f87b7124698a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:24.770924 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1100/2177] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:34:24 2022] DELETE /identity/v3/projects/76db848868cb417781b7bf896debf42c => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:24.775742 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2193e-ae87-4779-9cd6-926ff76fe2a1 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:24.776068 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2193e-ae87-4779-9cd6-926ff76fe2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:24.776344 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2193e-ae87-4779-9cd6-926ff76fe2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:24.797461 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-83d2193e-ae87-4779-9cd6-926ff76fe2a1 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:24.798704 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-83d2193e-ae87-4779-9cd6-926ff76fe2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:24.858673 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1078/2178] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:34:24.866474 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff2720-0151-485f-9a1a-63cd6dba3983 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:24.866874 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff2720-0151-485f-9a1a-63cd6dba3983 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:24.867352 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff2720-0151-485f-9a1a-63cd6dba3983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:24.877952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:24.889726 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-61ff2720-0151-485f-9a1a-63cd6dba3983 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:24.890986 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-61ff2720-0151-485f-9a1a-63cd6dba3983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:24.898774 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1101/2179] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:34:24.903042 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2193e-ae87-4779-9cd6-926ff76fe2a1 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:24.912727 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2193e-ae87-4779-9cd6-926ff76fe2a1 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:24.917756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e18e73d-9098-4c64-98c0-32218b2cb8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:24.918383 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18e73d-9098-4c64-98c0-32218b2cb8ff admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:24.918654 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18e73d-9098-4c64-98c0-32218b2cb8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:24.918921 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18e73d-9098-4c64-98c0-32218b2cb8ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:24.919554 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e18e73d-9098-4c64-98c0-32218b2cb8ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:34:24.919851 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e18e73d-9098-4c64-98c0-32218b2cb8ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:24.921448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e18e73d-9098-4c64-98c0-32218b2cb8ff admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:24.933426 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1079/2180] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:34:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:34:24.937608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff2720-0151-485f-9a1a-63cd6dba3983 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:24.947301 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff2720-0151-485f-9a1a-63cd6dba3983 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:24.952956 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e7ff40-c783-4cda-af25-1e4e8cd77d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:24.953670 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7ff40-c783-4cda-af25-1e4e8cd77d68 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:24.953951 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7ff40-c783-4cda-af25-1e4e8cd77d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:24.954256 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7ff40-c783-4cda-af25-1e4e8cd77d68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:24.954972 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e7ff40-c783-4cda-af25-1e4e8cd77d68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:24.957313 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e7ff40-c783-4cda-af25-1e4e8cd77d68 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:24.990584 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1102/2181] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:34:24 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:34:24.994880 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e18e73d-9098-4c64-98c0-32218b2cb8ff admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:25.006206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e18e73d-9098-4c64-98c0-32218b2cb8ff admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:25.018259 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:25.066012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce2c4b-70c5-4259-8fc9-deb3790db734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:25.066647 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce2c4b-70c5-4259-8fc9-deb3790db734 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:25.066928 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce2c4b-70c5-4259-8fc9-deb3790db734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:25.067202 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce2c4b-70c5-4259-8fc9-deb3790db734 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:25.067851 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fce2c4b-70c5-4259-8fc9-deb3790db734 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:25.069752 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fce2c4b-70c5-4259-8fc9-deb3790db734 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:25.090614 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1080/2182] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:34:24 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:34:25.095998 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e7ff40-c783-4cda-af25-1e4e8cd77d68 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:25.107947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e7ff40-c783-4cda-af25-1e4e8cd77d68 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:25.112645 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24a0ef-fbd9-4606-8ea4-415a58dcdc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:25.113185 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24a0ef-fbd9-4606-8ea4-415a58dcdc48 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:25.113460 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24a0ef-fbd9-4606-8ea4-415a58dcdc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:25.113730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24a0ef-fbd9-4606-8ea4-415a58dcdc48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:25.114396 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e24a0ef-fbd9-4606-8ea4-415a58dcdc48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:25.115977 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e24a0ef-fbd9-4606-8ea4-415a58dcdc48 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:25.125724 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1103/2183] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:34:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:34:25.129745 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce2c4b-70c5-4259-8fc9-deb3790db734 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:25.140738 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce2c4b-70c5-4259-8fc9-deb3790db734 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:25.145552 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b6dfc-745c-45a1-84a4-3267aa3a2a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:25.146201 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b6dfc-745c-45a1-84a4-3267aa3a2a7c admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:25.146488 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b6dfc-745c-45a1-84a4-3267aa3a2a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:25.146908 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b6dfc-745c-45a1-84a4-3267aa3a2a7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:25.147562 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b6dfc-745c-45a1-84a4-3267aa3a2a7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:25.149159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b6dfc-745c-45a1-84a4-3267aa3a2a7c admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:25.160768 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1081/2184] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:34:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:34:25.164998 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24a0ef-fbd9-4606-8ea4-415a58dcdc48 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:25.175725 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24a0ef-fbd9-4606-8ea4-415a58dcdc48 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:25.180521 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4f946-4a7f-438f-a2b8-00331d8b2e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:25.181182 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4f946-4a7f-438f-a2b8-00331d8b2e77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:25.181457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4f946-4a7f-438f-a2b8-00331d8b2e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:25.181728 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4f946-4a7f-438f-a2b8-00331d8b2e77 admin admin] PATH_INFO: `/v3/projects/d5804df4a7c54688953727e249ce63e1/users/d17e9dd3a1224bbcb38ab5135a918df1/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:25.196448 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4f946-4a7f-438f-a2b8-00331d8b2e77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5804df4a7c54688953727e249ce63e1, user_id=d17e9dd3a1224bbcb38ab5135a918df1, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:25.199183 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4f946-4a7f-438f-a2b8-00331d8b2e77 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:25.215354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1104/2185] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:34:25 2022] PUT /identity/v3/projects/d5804df4a7c54688953727e249ce63e1/users/d17e9dd3a1224bbcb38ab5135a918df1/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:25.220160 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e986aa7-65ac-4513-accd-e63c336a379f None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:25.220438 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e986aa7-65ac-4513-accd-e63c336a379f None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:25.220705 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e986aa7-65ac-4513-accd-e63c336a379f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:25.241687 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-5e986aa7-65ac-4513-accd-e63c336a379f None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:25.242941 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-5e986aa7-65ac-4513-accd-e63c336a379f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:25.309162 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1082/2186] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:34:25.318775 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4f946-4a7f-438f-a2b8-00331d8b2e77 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:25.320542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-16a4f946-4a7f-438f-a2b8-00331d8b2e77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:25.387570 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4f946-4a7f-438f-a2b8-00331d8b2e77 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:25.392055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1142b-c6b7-4397-9ac7-0fb82034f21a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:25.392684 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1142b-c6b7-4397-9ac7-0fb82034f21a service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:25.393087 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1142b-c6b7-4397-9ac7-0fb82034f21a service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:25.393336 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1142b-c6b7-4397-9ac7-0fb82034f21a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:25.404114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1142b-c6b7-4397-9ac7-0fb82034f21a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:25.405550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1142b-c6b7-4397-9ac7-0fb82034f21a service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:25.429201 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1105/2187] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:34:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:34:26.282580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81e7bd-7b0c-41d6-95fd-dc95d399418b None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:26.283680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81e7bd-7b0c-41d6-95fd-dc95d399418b None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:26.284134 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81e7bd-7b0c-41d6-95fd-dc95d399418b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:26.316870 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-be81e7bd-7b0c-41d6-95fd-dc95d399418b None None] MFA Rules not processed for user `d17e9dd3a1224bbcb38ab5135a918df1`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:26.318851 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-be81e7bd-7b0c-41d6-95fd-dc95d399418b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:26.436486 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1083/2188] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:34:26.457148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1142b-c6b7-4397-9ac7-0fb82034f21a service neutron] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:26.459249 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-20e1142b-c6b7-4397-9ac7-0fb82034f21a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:26.545784 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1142b-c6b7-4397-9ac7-0fb82034f21a service neutron] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:26.549807 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78ad09-dc02-4e42-894e-5b4f6b0a9073 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:26.550481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78ad09-dc02-4e42-894e-5b4f6b0a9073 service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:26.550768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78ad09-dc02-4e42-894e-5b4f6b0a9073 service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:26.551036 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78ad09-dc02-4e42-894e-5b4f6b0a9073 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:26.562484 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf78ad09-dc02-4e42-894e-5b4f6b0a9073 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:26.564427 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf78ad09-dc02-4e42-894e-5b4f6b0a9073 service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:26.575805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1106/2189] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:34:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:34:30.064384 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6354cbd-81d5-4d95-b9b7-5a70c7e18d69 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:30.064952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6354cbd-81d5-4d95-b9b7-5a70c7e18d69 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:30.064952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6354cbd-81d5-4d95-b9b7-5a70c7e18d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:30.085049 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-b6354cbd-81d5-4d95-b9b7-5a70c7e18d69 None None] MFA Rules not processed for user `d17e9dd3a1224bbcb38ab5135a918df1`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:30.086291 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-b6354cbd-81d5-4d95-b9b7-5a70c7e18d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:30.093689 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1084/2190] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:34:34.890650 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78ad09-dc02-4e42-894e-5b4f6b0a9073 service cinder] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:34.894009 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-cf78ad09-dc02-4e42-894e-5b4f6b0a9073 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:34.909172 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78ad09-dc02-4e42-894e-5b4f6b0a9073 service cinder] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:34.914163 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5ae52-7aac-4c9d-ab10-9d684f3c338e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:34.914860 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5ae52-7aac-4c9d-ab10-9d684f3c338e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:34.915200 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5ae52-7aac-4c9d-ab10-9d684f3c338e admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:34.915475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5ae52-7aac-4c9d-ab10-9d684f3c338e admin admin] PATH_INFO: `/v3/users/2360ae2b47f64ee58642ba8253d81aa3` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:34.916743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e5ae52-7aac-4c9d-ab10-9d684f3c338e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2360ae2b47f64ee58642ba8253d81aa3)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:34.919046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e5ae52-7aac-4c9d-ab10-9d684f3c338e admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:34.970790 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:34.980062 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-79e5ae52-7aac-4c9d-ab10-9d684f3c338e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2360ae2b47f64ee58642ba8253d81aa3'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:34.981223 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-79e5ae52-7aac-4c9d-ab10-9d684f3c338e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2360ae2b47f64ee58642ba8253d81aa3'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:34.991552 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-79e5ae52-7aac-4c9d-ab10-9d684f3c338e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2360ae2b47f64ee58642ba8253d81aa3'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:35.005309 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-79e5ae52-7aac-4c9d-ab10-9d684f3c338e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2360ae2b47f64ee58642ba8253d81aa3'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:35.006099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-79e5ae52-7aac-4c9d-ab10-9d684f3c338e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2360ae2b47f64ee58642ba8253d81aa3'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:35.020244 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1107/2191] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:34:34 2022] DELETE /identity/v3/users/2360ae2b47f64ee58642ba8253d81aa3 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:35.024735 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9989a5-e0c7-4670-8d2f-4de6b1d399a0 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:35.025263 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9989a5-e0c7-4670-8d2f-4de6b1d399a0 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:35.025371 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9989a5-e0c7-4670-8d2f-4de6b1d399a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:35.045378 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-aa9989a5-e0c7-4670-8d2f-4de6b1d399a0 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:35.046742 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-aa9989a5-e0c7-4670-8d2f-4de6b1d399a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:35.060190 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:35.112627 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1085/2192] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:34:35.122657 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5ae52-7aac-4c9d-ab10-9d684f3c338e admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:35.124056 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-79e5ae52-7aac-4c9d-ab10-9d684f3c338e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:35.188198 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5ae52-7aac-4c9d-ab10-9d684f3c338e admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:35.191932 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7bf419-9757-49a4-8f3d-edd1873c8e4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:35.192544 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7bf419-9757-49a4-8f3d-edd1873c8e4f service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:35.192826 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7bf419-9757-49a4-8f3d-edd1873c8e4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:35.193098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7bf419-9757-49a4-8f3d-edd1873c8e4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:35.202617 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7bf419-9757-49a4-8f3d-edd1873c8e4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:35.204130 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7bf419-9757-49a4-8f3d-edd1873c8e4f service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:35.214366 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1108/2193] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:34:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:34:35.520398 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9989a5-e0c7-4670-8d2f-4de6b1d399a0 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:35.521912 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-aa9989a5-e0c7-4670-8d2f-4de6b1d399a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:35.534873 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9989a5-e0c7-4670-8d2f-4de6b1d399a0 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:35.541223 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c16067-13a5-47fc-bc1c-2ad9eb02a1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:35.542447 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c16067-13a5-47fc-bc1c-2ad9eb02a1f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:35.542747 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c16067-13a5-47fc-bc1c-2ad9eb02a1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:35.543186 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c16067-13a5-47fc-bc1c-2ad9eb02a1f7 admin admin] PATH_INFO: `/v3/projects/ed4a241a593543d69cdd1fcfb757419c` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:35.544342 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c16067-13a5-47fc-bc1c-2ad9eb02a1f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed4a241a593543d69cdd1fcfb757419c)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:35.546290 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c16067-13a5-47fc-bc1c-2ad9eb02a1f7 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:35.595303 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-09c16067-13a5-47fc-bc1c-2ad9eb02a1f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed4a241a593543d69cdd1fcfb757419c', 'request_id': 'req-09c16067-13a5-47fc-bc1c-2ad9eb02a1f7'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:35.601960 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-09c16067-13a5-47fc-bc1c-2ad9eb02a1f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed4a241a593543d69cdd1fcfb757419c', 'request_id': 'req-09c16067-13a5-47fc-bc1c-2ad9eb02a1f7'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:35.608238 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-09c16067-13a5-47fc-bc1c-2ad9eb02a1f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed4a241a593543d69cdd1fcfb757419c', 'request_id': 'req-09c16067-13a5-47fc-bc1c-2ad9eb02a1f7'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:35.620026 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-09c16067-13a5-47fc-bc1c-2ad9eb02a1f7 admin admin] The token cache is being invalidate because project ed4a241a593543d69cdd1fcfb757419c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:34:35.620225 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-09c16067-13a5-47fc-bc1c-2ad9eb02a1f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:35.621037 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-09c16067-13a5-47fc-bc1c-2ad9eb02a1f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:35.622594 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1086/2194] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:34:35 2022] DELETE /identity/v3/projects/ed4a241a593543d69cdd1fcfb757419c => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:45.209763 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=40,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:45.616789 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=45,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:52.742898 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7bf419-9757-49a4-8f3d-edd1873c8e4f service neutron] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:52.744285 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-9c7bf419-9757-49a4-8f3d-edd1873c8e4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:52.757276 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7bf419-9757-49a4-8f3d-edd1873c8e4f service neutron] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:52.825641 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf8502-1418-4ae4-818c-2038c95a146e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:52.826312 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf8502-1418-4ae4-818c-2038c95a146e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:52.826589 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf8502-1418-4ae4-818c-2038c95a146e admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:52.826863 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf8502-1418-4ae4-818c-2038c95a146e admin admin] PATH_INFO: `/v3/users/ab41daa8d6624064ae4e4a8009905667` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:52.827991 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf8502-1418-4ae4-818c-2038c95a146e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab41daa8d6624064ae4e4a8009905667)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:52.829890 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf8502-1418-4ae4-818c-2038c95a146e admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:52.862542 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01637c-9a12-41f1-98af-7dfd48cdba13 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:52.863040 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01637c-9a12-41f1-98af-7dfd48cdba13 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:52.863130 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01637c-9a12-41f1-98af-7dfd48cdba13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:52.882824 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-6c01637c-9a12-41f1-98af-7dfd48cdba13 None None] MFA Rules not processed for user `d17e9dd3a1224bbcb38ab5135a918df1`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:52.884136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-6c01637c-9a12-41f1-98af-7dfd48cdba13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:52.886756 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-76cf8502-1418-4ae4-818c-2038c95a146e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab41daa8d6624064ae4e4a8009905667'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:52.887424 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-76cf8502-1418-4ae4-818c-2038c95a146e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab41daa8d6624064ae4e4a8009905667'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:52.895396 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-76cf8502-1418-4ae4-818c-2038c95a146e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab41daa8d6624064ae4e4a8009905667'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:52.909160 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-76cf8502-1418-4ae4-818c-2038c95a146e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab41daa8d6624064ae4e4a8009905667'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:52.909857 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-76cf8502-1418-4ae4-818c-2038c95a146e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab41daa8d6624064ae4e4a8009905667'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:52.924003 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1109/2195] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:34:52 2022] DELETE /identity/v3/users/ab41daa8d6624064ae4e4a8009905667 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:34:52.928246 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6e163-2a6f-4d05-8b81-2141dac1465d None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:52.928528 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6e163-2a6f-4d05-8b81-2141dac1465d None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:52.928806 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6e163-2a6f-4d05-8b81-2141dac1465d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:52.943002 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1087/2196] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:34:52.947602 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-ffc6e163-2a6f-4d05-8b81-2141dac1465d None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:52.948802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-ffc6e163-2a6f-4d05-8b81-2141dac1465d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:53.009190 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1110/2197] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:34:53.018980 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c01637c-9a12-41f1-98af-7dfd48cdba13 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:53.020441 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-6c01637c-9a12-41f1-98af-7dfd48cdba13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:53.082011 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c01637c-9a12-41f1-98af-7dfd48cdba13 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:53.143733 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054518d9-c96a-496a-b391-2510b7b29949 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:53.144378 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054518d9-c96a-496a-b391-2510b7b29949 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:53.144786 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054518d9-c96a-496a-b391-2510b7b29949 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:53.145065 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054518d9-c96a-496a-b391-2510b7b29949 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:53.155603 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054518d9-c96a-496a-b391-2510b7b29949 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:53.157035 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054518d9-c96a-496a-b391-2510b7b29949 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:53.167180 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1088/2198] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:34:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:34:53.509622 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6e163-2a6f-4d05-8b81-2141dac1465d None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:53.511420 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-ffc6e163-2a6f-4d05-8b81-2141dac1465d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:53.533297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6e163-2a6f-4d05-8b81-2141dac1465d None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:53.543402 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16da64e2-9962-4b08-afd3-82e28b90fd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:53.544893 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da64e2-9962-4b08-afd3-82e28b90fd31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:53.545510 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da64e2-9962-4b08-afd3-82e28b90fd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:53.545809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da64e2-9962-4b08-afd3-82e28b90fd31 admin admin] PATH_INFO: `/v3/projects/6c0c5298b56944e1a342ee54ecdf3ceb` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:53.547070 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16da64e2-9962-4b08-afd3-82e28b90fd31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c0c5298b56944e1a342ee54ecdf3ceb)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:53.549084 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16da64e2-9962-4b08-afd3-82e28b90fd31 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:53.572525 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:53.613611 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-16da64e2-9962-4b08-afd3-82e28b90fd31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c0c5298b56944e1a342ee54ecdf3ceb', 'request_id': 'req-16da64e2-9962-4b08-afd3-82e28b90fd31'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:53.619859 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-16da64e2-9962-4b08-afd3-82e28b90fd31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c0c5298b56944e1a342ee54ecdf3ceb', 'request_id': 'req-16da64e2-9962-4b08-afd3-82e28b90fd31'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:53.628272 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-16da64e2-9962-4b08-afd3-82e28b90fd31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c0c5298b56944e1a342ee54ecdf3ceb', 'request_id': 'req-16da64e2-9962-4b08-afd3-82e28b90fd31'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:53.640529 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-16da64e2-9962-4b08-afd3-82e28b90fd31 admin admin] The token cache is being invalidate because project 6c0c5298b56944e1a342ee54ecdf3ceb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:34:53.640814 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-16da64e2-9962-4b08-afd3-82e28b90fd31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:53.641528 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-16da64e2-9962-4b08-afd3-82e28b90fd31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:34:53.643080 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1111/2199] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:34:53 2022] DELETE /identity/v3/projects/6c0c5298b56944e1a342ee54ecdf3ceb => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:53.647958 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29681e-de0c-489e-b014-3b5d73fefe37 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:53.648432 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29681e-de0c-489e-b014-3b5d73fefe37 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:53.648707 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29681e-de0c-489e-b014-3b5d73fefe37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:53.669337 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-fb29681e-de0c-489e-b014-3b5d73fefe37 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:53.670586 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-fb29681e-de0c-489e-b014-3b5d73fefe37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:53.741204 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1089/2200] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:34:53.750271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dcef02-0df8-4d59-bcdb-ef5a608a3c79 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:53.750817 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dcef02-0df8-4d59-bcdb-ef5a608a3c79 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:53.750817 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dcef02-0df8-4d59-bcdb-ef5a608a3c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:53.772812 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-a0dcef02-0df8-4d59-bcdb-ef5a608a3c79 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:53.774087 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-a0dcef02-0df8-4d59-bcdb-ef5a608a3c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:53.786001 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1112/2201] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:34:53.787215 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29681e-de0c-489e-b014-3b5d73fefe37 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:53.793698 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:53.800303 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29681e-de0c-489e-b014-3b5d73fefe37 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:53.808782 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d955e-6f69-40dd-aff5-aed712ba1bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:53.810196 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d955e-6f69-40dd-aff5-aed712ba1bfa admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:53.810789 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d955e-6f69-40dd-aff5-aed712ba1bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:53.810933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d955e-6f69-40dd-aff5-aed712ba1bfa admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:53.812225 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d955e-6f69-40dd-aff5-aed712ba1bfa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:34:53.812225 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d955e-6f69-40dd-aff5-aed712ba1bfa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:53.814748 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d955e-6f69-40dd-aff5-aed712ba1bfa admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:53.832548 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1090/2202] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:34:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:34:53.837965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dcef02-0df8-4d59-bcdb-ef5a608a3c79 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:53.858464 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dcef02-0df8-4d59-bcdb-ef5a608a3c79 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:53.858464 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144aa78c-78b9-4ab3-a9c4-4f2f7f59402c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:53.858464 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144aa78c-78b9-4ab3-a9c4-4f2f7f59402c admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:53.858464 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144aa78c-78b9-4ab3-a9c4-4f2f7f59402c admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:53.858809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144aa78c-78b9-4ab3-a9c4-4f2f7f59402c admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:53.858809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144aa78c-78b9-4ab3-a9c4-4f2f7f59402c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:53.859237 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144aa78c-78b9-4ab3-a9c4-4f2f7f59402c admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:53.898044 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1113/2203] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:34:53 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:34:53.899115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d955e-6f69-40dd-aff5-aed712ba1bfa admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:53.912004 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d955e-6f69-40dd-aff5-aed712ba1bfa admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:53.980362 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae029fb2-f9ec-4add-b0f3-686db9e87325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:53.981947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae029fb2-f9ec-4add-b0f3-686db9e87325 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:53.982299 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae029fb2-f9ec-4add-b0f3-686db9e87325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:53.982729 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae029fb2-f9ec-4add-b0f3-686db9e87325 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:53.983472 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae029fb2-f9ec-4add-b0f3-686db9e87325 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:53.986313 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae029fb2-f9ec-4add-b0f3-686db9e87325 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:54.014089 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1091/2204] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:34:53 2022] POST /identity/v3/users => generated 292 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:34:54.018378 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144aa78c-78b9-4ab3-a9c4-4f2f7f59402c admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:54.029361 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144aa78c-78b9-4ab3-a9c4-4f2f7f59402c admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:54.033917 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f3f18-647c-4452-9d17-dc633e115f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:54.034537 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f3f18-647c-4452-9d17-dc633e115f3c admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:54.034811 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f3f18-647c-4452-9d17-dc633e115f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:54.035079 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f3f18-647c-4452-9d17-dc633e115f3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:54.035712 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9f3f18-647c-4452-9d17-dc633e115f3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:54.037316 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9f3f18-647c-4452-9d17-dc633e115f3c admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:54.047620 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1114/2205] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:34:54.051544 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae029fb2-f9ec-4add-b0f3-686db9e87325 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:54.062372 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae029fb2-f9ec-4add-b0f3-686db9e87325 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:54.066975 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a9f2a-4967-4903-859a-457f131f174a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:54.067559 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a9f2a-4967-4903-859a-457f131f174a admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:54.067835 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a9f2a-4967-4903-859a-457f131f174a admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:54.068106 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a9f2a-4967-4903-859a-457f131f174a admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:54.068737 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9a9f2a-4967-4903-859a-457f131f174a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:54.070342 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9a9f2a-4967-4903-859a-457f131f174a admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:54.080699 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1092/2206] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:34:54.084649 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f3f18-647c-4452-9d17-dc633e115f3c admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:54.094723 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f3f18-647c-4452-9d17-dc633e115f3c admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:54.099642 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d202336-a7ce-49ac-b431-1841e6b3f943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:54.100543 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d202336-a7ce-49ac-b431-1841e6b3f943 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:54.100819 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d202336-a7ce-49ac-b431-1841e6b3f943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:54.101117 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d202336-a7ce-49ac-b431-1841e6b3f943 admin admin] PATH_INFO: `/v3/projects/6bb20c7af7544a06adaa9b5e47b03cd1/users/681edd9335a644a69a497a5c528920a8/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:54.116167 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d202336-a7ce-49ac-b431-1841e6b3f943 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bb20c7af7544a06adaa9b5e47b03cd1, user_id=681edd9335a644a69a497a5c528920a8, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:54.118844 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d202336-a7ce-49ac-b431-1841e6b3f943 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:54.131726 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1115/2207] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:34:54 2022] PUT /identity/v3/projects/6bb20c7af7544a06adaa9b5e47b03cd1/users/681edd9335a644a69a497a5c528920a8/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:54.136636 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fec708-d1a3-4ebc-8c94-d03fac66545c None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:54.136919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fec708-d1a3-4ebc-8c94-d03fac66545c None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:54.137243 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fec708-d1a3-4ebc-8c94-d03fac66545c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:54.156173 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-61fec708-d1a3-4ebc-8c94-d03fac66545c None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:54.157363 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-61fec708-d1a3-4ebc-8c94-d03fac66545c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:54.237747 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1093/2208] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:34:54.247349 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d202336-a7ce-49ac-b431-1841e6b3f943 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:54.249764 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-3d202336-a7ce-49ac-b431-1841e6b3f943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:54.309817 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d202336-a7ce-49ac-b431-1841e6b3f943 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:54.313580 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d901884-4626-4593-bcca-885f10c46499 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:54.314225 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d901884-4626-4593-bcca-885f10c46499 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:54.314504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d901884-4626-4593-bcca-885f10c46499 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:54.314776 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d901884-4626-4593-bcca-885f10c46499 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:54.324681 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d901884-4626-4593-bcca-885f10c46499 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:54.326170 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d901884-4626-4593-bcca-885f10c46499 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:54.335587 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1116/2209] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:34:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:34:59.136739 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd28f02-b6e6-4313-b933-da0af9ceb9b8 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:59.138014 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd28f02-b6e6-4313-b933-da0af9ceb9b8 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:59.138014 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd28f02-b6e6-4313-b933-da0af9ceb9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:59.172559 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-0fd28f02-b6e6-4313-b933-da0af9ceb9b8 None None] MFA Rules not processed for user `681edd9335a644a69a497a5c528920a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:34:59.173797 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-0fd28f02-b6e6-4313-b933-da0af9ceb9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:34:59.289466 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:59.295804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1094/2210] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:34:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:34:59.299853 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d901884-4626-4593-bcca-885f10c46499 service neutron] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:59.309619 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d901884-4626-4593-bcca-885f10c46499 service neutron] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:59.320165 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44812db-9d99-4937-bda9-8cb517ee2c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:59.321238 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44812db-9d99-4937-bda9-8cb517ee2c25 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:59.321735 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44812db-9d99-4937-bda9-8cb517ee2c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:59.322188 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44812db-9d99-4937-bda9-8cb517ee2c25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:59.323242 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44812db-9d99-4937-bda9-8cb517ee2c25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:59.328561 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44812db-9d99-4937-bda9-8cb517ee2c25 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:59.392816 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1117/2211] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:34:59 2022] POST /identity/v3/projects => generated 385 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:34:59.393937 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd28f02-b6e6-4313-b933-da0af9ceb9b8 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:59.429991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd28f02-b6e6-4313-b933-da0af9ceb9b8 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:59.542833 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ea68a4-5210-4ca1-b69a-4153951e743d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:59.543918 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ea68a4-5210-4ca1-b69a-4153951e743d admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:59.544196 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ea68a4-5210-4ca1-b69a-4153951e743d admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:59.544464 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ea68a4-5210-4ca1-b69a-4153951e743d admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:59.545419 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ea68a4-5210-4ca1-b69a-4153951e743d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:59.547090 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ea68a4-5210-4ca1-b69a-4153951e743d admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:59.588994 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1095/2212] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:34:59 2022] POST /identity/v3/users => generated 299 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:34:59.593149 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44812db-9d99-4937-bda9-8cb517ee2c25 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:59.638099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44812db-9d99-4937-bda9-8cb517ee2c25 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:59.649512 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c13cd-c5b7-4023-a820-ddf1dc6c8d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:59.651756 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c13cd-c5b7-4023-a820-ddf1dc6c8d66 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:59.652190 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c13cd-c5b7-4023-a820-ddf1dc6c8d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:59.652506 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c13cd-c5b7-4023-a820-ddf1dc6c8d66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:59.654674 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c13cd-c5b7-4023-a820-ddf1dc6c8d66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:59.656379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c13cd-c5b7-4023-a820-ddf1dc6c8d66 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:59.685782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1118/2213] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:34:59.690232 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ea68a4-5210-4ca1-b69a-4153951e743d admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:59.705873 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ea68a4-5210-4ca1-b69a-4153951e743d admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:59.717013 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e0e0e-147b-4e8f-af90-80a63976e5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:59.717895 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e0e0e-147b-4e8f-af90-80a63976e5b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:59.718429 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e0e0e-147b-4e8f-af90-80a63976e5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:59.718879 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e0e0e-147b-4e8f-af90-80a63976e5b4 admin admin] PATH_INFO: `/v3/projects/bbb6530b068f4adba1992dd64f303e79/users/80c4c3b2a04f44b49021fb61b4c53826/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:59.807244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e0e0e-147b-4e8f-af90-80a63976e5b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbb6530b068f4adba1992dd64f303e79, user_id=80c4c3b2a04f44b49021fb61b4c53826, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:34:59.839359 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e0e0e-147b-4e8f-af90-80a63976e5b4 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:34:59.866859 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1096/2214] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:34:59 2022] PUT /identity/v3/projects/bbb6530b068f4adba1992dd64f303e79/users/80c4c3b2a04f44b49021fb61b4c53826/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:34:59.867976 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c13cd-c5b7-4023-a820-ddf1dc6c8d66 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:34:59.881407 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c13cd-c5b7-4023-a820-ddf1dc6c8d66 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:34:59.887878 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:34:59.996202 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37a49a-5e61-49dc-bdd0-cfe25950b9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:34:59.997635 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37a49a-5e61-49dc-bdd0-cfe25950b9c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:34:59.998160 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37a49a-5e61-49dc-bdd0-cfe25950b9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:34:59.998602 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37a49a-5e61-49dc-bdd0-cfe25950b9c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:34:59.999421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e37a49a-5e61-49dc-bdd0-cfe25950b9c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:35:00.001289 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e37a49a-5e61-49dc-bdd0-cfe25950b9c7 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:35:00.029650 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e0e0e-147b-4e8f-af90-80a63976e5b4 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:35:00.046238 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1119/2215] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:35:00.049744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e0e0e-147b-4e8f-af90-80a63976e5b4 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:35:00.056561 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b026ae-5861-409d-80fe-1e33b17d7b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:35:00.057271 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b026ae-5861-409d-80fe-1e33b17d7b86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:00.057549 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b026ae-5861-409d-80fe-1e33b17d7b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:00.057851 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b026ae-5861-409d-80fe-1e33b17d7b86 admin admin] PATH_INFO: `/v3/projects/bbb6530b068f4adba1992dd64f303e79/users/80c4c3b2a04f44b49021fb61b4c53826/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:00.060879 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b026ae-5861-409d-80fe-1e33b17d7b86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbb6530b068f4adba1992dd64f303e79, user_id=80c4c3b2a04f44b49021fb61b4c53826, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:35:00.063605 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b026ae-5861-409d-80fe-1e33b17d7b86 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:35:00.087687 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1097/2216] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:35:00 2022] PUT /identity/v3/projects/bbb6530b068f4adba1992dd64f303e79/users/80c4c3b2a04f44b49021fb61b4c53826/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:35:02.658057 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9089a-6bf5-4143-8203-af571754a28f None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:02.658501 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9089a-6bf5-4143-8203-af571754a28f None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:02.658724 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9089a-6bf5-4143-8203-af571754a28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:02.696265 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-efc9089a-6bf5-4143-8203-af571754a28f None None] MFA Rules not processed for user `5085a918c6b74488b3dd70c1fb257381`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:35:02.697801 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-efc9089a-6bf5-4143-8203-af571754a28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:35:02.820152 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1120/2217] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:35:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:35:05.989949 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92722e1-0779-4fb1-9a08-4ebb03edf74e None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:05.990464 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92722e1-0779-4fb1-9a08-4ebb03edf74e None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:05.990632 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92722e1-0779-4fb1-9a08-4ebb03edf74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:06.014700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-f92722e1-0779-4fb1-9a08-4ebb03edf74e None None] MFA Rules not processed for user `b486a094550541729c19c8713a648243`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:35:06.016214 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-f92722e1-0779-4fb1-9a08-4ebb03edf74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:35:06.093319 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1098/2218] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:35:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:35:09.217190 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e4ccd-3673-440e-aa05-9bd176b073e9 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:09.217649 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e4ccd-3673-440e-aa05-9bd176b073e9 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:09.218004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e4ccd-3673-440e-aa05-9bd176b073e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:09.265256 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-341e4ccd-3673-440e-aa05-9bd176b073e9 None None] MFA Rules not processed for user `80c4c3b2a04f44b49021fb61b4c53826`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:35:09.266779 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-341e4ccd-3673-440e-aa05-9bd176b073e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:35:09.394888 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1121/2219] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:35:09 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:35:09.404407 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92722e1-0779-4fb1-9a08-4ebb03edf74e None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:35:09.406164 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-f92722e1-0779-4fb1-9a08-4ebb03edf74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:35:09.469555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92722e1-0779-4fb1-9a08-4ebb03edf74e None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:35:09.473527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51aba4e-f6cf-4579-8102-d11a4bff703f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41b242fbe2504f8cac26ce4ff46f5390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:35:09.474269 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51aba4e-f6cf-4579-8102-d11a4bff703f service glance] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:09.474661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51aba4e-f6cf-4579-8102-d11a4bff703f service glance] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:09.475028 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51aba4e-f6cf-4579-8102-d11a4bff703f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:09.487085 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:35:09.572774 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51aba4e-f6cf-4579-8102-d11a4bff703f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:35:09.574269 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51aba4e-f6cf-4579-8102-d11a4bff703f service glance] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:35:09.597164 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1099/2220] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:35:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:35:09.912933 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcba3db-a59c-4176-bd08-701334556ea5 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:09.913520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcba3db-a59c-4176-bd08-701334556ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:09.914572 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcba3db-a59c-4176-bd08-701334556ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:09.946703 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-adcba3db-a59c-4176-bd08-701334556ea5 None None] MFA Rules not processed for user `9158557f1a2947e2815dc8d586d8b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:35:09.948194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-adcba3db-a59c-4176-bd08-701334556ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:35:10.005342 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:35:10.053252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1122/2221] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Sep 26 10:35:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:35:10.112822 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51aba4e-f6cf-4579-8102-d11a4bff703f service glance] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:35:10.115467 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-b51aba4e-f6cf-4579-8102-d11a4bff703f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:35:10.260329 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51aba4e-f6cf-4579-8102-d11a4bff703f service glance] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:35:10.275875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee597ea4-d11f-426c-a577-a568337b5655 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e6be34fc93a4551a83e5b8380188a6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:35:10.276892 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee597ea4-d11f-426c-a577-a568337b5655 service swift] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:10.277398 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee597ea4-d11f-426c-a577-a568337b5655 service swift] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:10.278387 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee597ea4-d11f-426c-a577-a568337b5655 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:10.298493 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee597ea4-d11f-426c-a577-a568337b5655 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:35:10.300645 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee597ea4-d11f-426c-a577-a568337b5655 service swift] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:35:10.315644 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1100/2222] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 10:35:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 10:35:20.039469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=9 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:35:20.307548 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=54 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:35:22.053748 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e0571-0012-43e1-a85f-e81d1891ee3a None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:22.054386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e0571-0012-43e1-a85f-e81d1891ee3a None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:22.054386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e0571-0012-43e1-a85f-e81d1891ee3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:22.073714 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-132e0571-0012-43e1-a85f-e81d1891ee3a None None] MFA Rules not processed for user `681edd9335a644a69a497a5c528920a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:35:22.076001 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-132e0571-0012-43e1-a85f-e81d1891ee3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:35:22.083476 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1123/2223] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:35:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:35:22.533129 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88589888-6962-4ce3-96c8-0d0c5437421c None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:22.533644 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88589888-6962-4ce3-96c8-0d0c5437421c None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:22.533736 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88589888-6962-4ce3-96c8-0d0c5437421c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:22.563762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-88589888-6962-4ce3-96c8-0d0c5437421c None None] MFA Rules not processed for user `9158557f1a2947e2815dc8d586d8b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:35:22.564873 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-88589888-6962-4ce3-96c8-0d0c5437421c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:35:22.572950 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1101/2224] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Sep 26 10:35:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:35:22.586565 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e0571-0012-43e1-a85f-e81d1891ee3a None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:35:22.599455 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e0571-0012-43e1-a85f-e81d1891ee3a None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:35:22.605440 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4795385-737d-477d-a829-ba353c5e54d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e6be34fc93a4551a83e5b8380188a6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:35:22.606714 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4795385-737d-477d-a829-ba353c5e54d4 service swift] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:22.607619 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4795385-737d-477d-a829-ba353c5e54d4 service swift] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:22.608203 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4795385-737d-477d-a829-ba353c5e54d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:22.618669 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4795385-737d-477d-a829-ba353c5e54d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:35:22.620434 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4795385-737d-477d-a829-ba353c5e54d4 service swift] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:35:22.629918 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1124/2225] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 10:35:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 10:35:27.403385 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cfe838-604b-425f-87cf-bfa30e75ca6d None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:27.404959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cfe838-604b-425f-87cf-bfa30e75ca6d None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:27.405405 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cfe838-604b-425f-87cf-bfa30e75ca6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:27.434928 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-a0cfe838-604b-425f-87cf-bfa30e75ca6d None None] MFA Rules not processed for user `9158557f1a2947e2815dc8d586d8b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:35:27.437134 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-a0cfe838-604b-425f-87cf-bfa30e75ca6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:35:27.454104 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1102/2226] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Sep 26 10:35:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:35:27.467247 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4795385-737d-477d-a829-ba353c5e54d4 service swift] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:35:27.480677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4795385-737d-477d-a829-ba353c5e54d4 service swift] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:35:27.501376 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f3084d-9f7e-46a2-88ee-dfc5883413e7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e6be34fc93a4551a83e5b8380188a6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:35:27.504143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3084d-9f7e-46a2-88ee-dfc5883413e7 service swift] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:27.504661 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3084d-9f7e-46a2-88ee-dfc5883413e7 service swift] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:27.505642 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3084d-9f7e-46a2-88ee-dfc5883413e7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:27.520179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f3084d-9f7e-46a2-88ee-dfc5883413e7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:35:27.521993 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f3084d-9f7e-46a2-88ee-dfc5883413e7 service swift] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:35:27.535099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1125/2227] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 10:35:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 10:35:37.424628 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=6 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:35:37.530122 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=15 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:35:41.618799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ef064-52df-49a5-911e-be0bdfa3035e None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:41.619350 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ef064-52df-49a5-911e-be0bdfa3035e None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:41.619350 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ef064-52df-49a5-911e-be0bdfa3035e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:41.639354 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-cc3ef064-52df-49a5-911e-be0bdfa3035e None None] MFA Rules not processed for user `9158557f1a2947e2815dc8d586d8b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:35:41.640565 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-cc3ef064-52df-49a5-911e-be0bdfa3035e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:35:41.648612 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1103/2228] 10.222.0.49 () {58 vars in 967 bytes} [Mon Sep 26 10:35:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:35:41.661746 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f3084d-9f7e-46a2-88ee-dfc5883413e7 service swift] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:35:41.673695 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f3084d-9f7e-46a2-88ee-dfc5883413e7 service swift] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:35:41.678805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9a6e1-cd71-4d6f-a8ed-b672192bbe2d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e6be34fc93a4551a83e5b8380188a6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:35:41.679402 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9a6e1-cd71-4d6f-a8ed-b672192bbe2d service swift] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:41.679681 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9a6e1-cd71-4d6f-a8ed-b672192bbe2d service swift] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:41.679948 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9a6e1-cd71-4d6f-a8ed-b672192bbe2d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:41.688409 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9a6e1-cd71-4d6f-a8ed-b672192bbe2d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:35:41.689967 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9a6e1-cd71-4d6f-a8ed-b672192bbe2d service swift] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:35:41.697988 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1126/2229] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 10:35:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 10:35:43.695735 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff31f35-8437-468c-a548-978ee34a8824 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:43.696327 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff31f35-8437-468c-a548-978ee34a8824 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:43.696327 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff31f35-8437-468c-a548-978ee34a8824 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:43.697612 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1104/2230] 10.222.0.49 () {54 vars in 921 bytes} [Mon Sep 26 10:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:35:43.709969 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e48f03-d009-439b-bd2c-262930575c89 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:43.710833 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e48f03-d009-439b-bd2c-262930575c89 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:43.711287 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e48f03-d009-439b-bd2c-262930575c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:43.737910 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-81e48f03-d009-439b-bd2c-262930575c89 None None] MFA Rules not processed for user `84b02e7351914cabb67462454b328d05`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:35:43.740152 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-81e48f03-d009-439b-bd2c-262930575c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:35:43.774322 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff31f35-8437-468c-a548-978ee34a8824 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:35:43.776380 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-3ff31f35-8437-468c-a548-978ee34a8824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:35:43.812082 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1127/2231] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Sep 26 10:35:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:35:43.823378 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e48f03-d009-439b-bd2c-262930575c89 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:35:43.826442 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-81e48f03-d009-439b-bd2c-262930575c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:35:43.839153 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e48f03-d009-439b-bd2c-262930575c89 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:35:43.844323 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715de1d-4273-4446-a76a-124f72a1ae78 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84b02e7351914cabb67462454b328d05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:35:43.844926 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715de1d-4273-4446-a76a-124f72a1ae78 service placement] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:43.845207 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715de1d-4273-4446-a76a-124f72a1ae78 service placement] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:43.845477 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715de1d-4273-4446-a76a-124f72a1ae78 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:43.854279 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9715de1d-4273-4446-a76a-124f72a1ae78 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:35:43.855936 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9715de1d-4273-4446-a76a-124f72a1ae78 service placement] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:35:43.856455 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff31f35-8437-468c-a548-978ee34a8824 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:35:43.860814 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af43473c-158e-4dd8-a4de-dc1989bf0738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:35:43.861525 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43473c-158e-4dd8-a4de-dc1989bf0738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:43.861751 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43473c-158e-4dd8-a4de-dc1989bf0738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:43.862007 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43473c-158e-4dd8-a4de-dc1989bf0738 admin admin] PATH_INFO: `/v3/users/d17e9dd3a1224bbcb38ab5135a918df1` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:43.863150 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af43473c-158e-4dd8-a4de-dc1989bf0738 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d17e9dd3a1224bbcb38ab5135a918df1)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:35:43.865041 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af43473c-158e-4dd8-a4de-dc1989bf0738 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:35:43.866181 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1128/2232] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:35:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:35:43.916324 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-af43473c-158e-4dd8-a4de-dc1989bf0738 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd17e9dd3a1224bbcb38ab5135a918df1'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:35:43.916993 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-af43473c-158e-4dd8-a4de-dc1989bf0738 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd17e9dd3a1224bbcb38ab5135a918df1'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:35:43.917607 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-af43473c-158e-4dd8-a4de-dc1989bf0738 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd17e9dd3a1224bbcb38ab5135a918df1'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:35:43.924457 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-af43473c-158e-4dd8-a4de-dc1989bf0738 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd17e9dd3a1224bbcb38ab5135a918df1'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:35:43.936589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-af43473c-158e-4dd8-a4de-dc1989bf0738 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd17e9dd3a1224bbcb38ab5135a918df1'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:35:43.949210 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1105/2233] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:35:43 2022] DELETE /identity/v3/users/d17e9dd3a1224bbcb38ab5135a918df1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:35:44.126411 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715de1d-4273-4446-a76a-124f72a1ae78 service placement] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:35:44.137375 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715de1d-4273-4446-a76a-124f72a1ae78 service placement] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:35:44.195945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48bbfe-fe5a-45c2-9535-392e0256e0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:35:44.196572 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48bbfe-fe5a-45c2-9535-392e0256e0f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:44.196850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48bbfe-fe5a-45c2-9535-392e0256e0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:44.197123 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48bbfe-fe5a-45c2-9535-392e0256e0f0 admin admin] PATH_INFO: `/v3/projects/d5804df4a7c54688953727e249ce63e1` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:44.198257 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e48bbfe-fe5a-45c2-9535-392e0256e0f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5804df4a7c54688953727e249ce63e1)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:35:44.200189 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e48bbfe-fe5a-45c2-9535-392e0256e0f0 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:35:44.248867 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-2e48bbfe-fe5a-45c2-9535-392e0256e0f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5804df4a7c54688953727e249ce63e1', 'request_id': 'req-2e48bbfe-fe5a-45c2-9535-392e0256e0f0'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:35:44.254952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-2e48bbfe-fe5a-45c2-9535-392e0256e0f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5804df4a7c54688953727e249ce63e1', 'request_id': 'req-2e48bbfe-fe5a-45c2-9535-392e0256e0f0'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:35:44.262110 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-2e48bbfe-fe5a-45c2-9535-392e0256e0f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5804df4a7c54688953727e249ce63e1', 'request_id': 'req-2e48bbfe-fe5a-45c2-9535-392e0256e0f0'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:35:44.268944 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:35:44.273000 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-2e48bbfe-fe5a-45c2-9535-392e0256e0f0 admin admin] The token cache is being invalidate because project d5804df4a7c54688953727e249ce63e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:35:44.273339 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-2e48bbfe-fe5a-45c2-9535-392e0256e0f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:35:44.274298 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-2e48bbfe-fe5a-45c2-9535-392e0256e0f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:35:44.276092 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1129/2234] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:35:44 2022] DELETE /identity/v3/projects/d5804df4a7c54688953727e249ce63e1 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:35:44.281037 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd57f37-89dc-49ac-a6a1-9ae811f3d2c0 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:44.281313 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd57f37-89dc-49ac-a6a1-9ae811f3d2c0 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:44.281583 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd57f37-89dc-49ac-a6a1-9ae811f3d2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:44.301602 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-7bd57f37-89dc-49ac-a6a1-9ae811f3d2c0 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:35:44.302837 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-7bd57f37-89dc-49ac-a6a1-9ae811f3d2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:35:44.360735 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1106/2235] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:35:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:35:44.368241 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3a227-304e-4a9b-8d7e-3a9e7fc4479d None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:44.368489 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3a227-304e-4a9b-8d7e-3a9e7fc4479d None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:44.368787 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3a227-304e-4a9b-8d7e-3a9e7fc4479d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:44.388861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-13e3a227-304e-4a9b-8d7e-3a9e7fc4479d None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:35:44.390095 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-13e3a227-304e-4a9b-8d7e-3a9e7fc4479d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:35:44.398979 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1130/2236] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:35:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:35:44.403458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd57f37-89dc-49ac-a6a1-9ae811f3d2c0 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:35:44.414868 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd57f37-89dc-49ac-a6a1-9ae811f3d2c0 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:35:44.420477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148bdb43-172e-45c9-bdfc-ae1ede110eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:35:44.421106 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148bdb43-172e-45c9-bdfc-ae1ede110eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:44.421386 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148bdb43-172e-45c9-bdfc-ae1ede110eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:44.421660 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148bdb43-172e-45c9-bdfc-ae1ede110eb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:44.422497 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148bdb43-172e-45c9-bdfc-ae1ede110eb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:35:44.422799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148bdb43-172e-45c9-bdfc-ae1ede110eb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:35:44.424566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148bdb43-172e-45c9-bdfc-ae1ede110eb1 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:35:44.434648 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:35:44.441667 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1107/2237] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:35:44.443294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3a227-304e-4a9b-8d7e-3a9e7fc4479d None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:35:44.453512 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3a227-304e-4a9b-8d7e-3a9e7fc4479d None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:35:44.461631 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd227907-74eb-4690-8403-6a7cd296a654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:35:44.462286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd227907-74eb-4690-8403-6a7cd296a654 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:44.462564 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd227907-74eb-4690-8403-6a7cd296a654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:44.462849 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd227907-74eb-4690-8403-6a7cd296a654 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:44.463472 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd227907-74eb-4690-8403-6a7cd296a654 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:35:44.465960 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd227907-74eb-4690-8403-6a7cd296a654 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:35:44.497032 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148bdb43-172e-45c9-bdfc-ae1ede110eb1 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:35:44.497818 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1131/2238] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:35:44 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:35:44.501525 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd227907-74eb-4690-8403-6a7cd296a654 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:35:44.502107 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-148bdb43-172e-45c9-bdfc-ae1ede110eb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:35:44.584812 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd227907-74eb-4690-8403-6a7cd296a654 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:35:44.590768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e764c4-9c32-45ec-b9c5-4350aad3770c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:35:44.591370 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e764c4-9c32-45ec-b9c5-4350aad3770c admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:44.591643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e764c4-9c32-45ec-b9c5-4350aad3770c admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:44.591913 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e764c4-9c32-45ec-b9c5-4350aad3770c admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:44.592543 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e764c4-9c32-45ec-b9c5-4350aad3770c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:35:44.593137 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148bdb43-172e-45c9-bdfc-ae1ede110eb1 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:35:44.594930 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e764c4-9c32-45ec-b9c5-4350aad3770c admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:35:44.618796 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1132/2239] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:35:44 2022] POST /identity/v3/users => generated 283 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:35:44.622788 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e764c4-9c32-45ec-b9c5-4350aad3770c admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:35:44.634693 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e764c4-9c32-45ec-b9c5-4350aad3770c admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:35:44.639520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6363cc27-2fc3-4bcc-91ff-285be3ae7fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:35:44.640205 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363cc27-2fc3-4bcc-91ff-285be3ae7fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:44.640545 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363cc27-2fc3-4bcc-91ff-285be3ae7fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:44.640876 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363cc27-2fc3-4bcc-91ff-285be3ae7fd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:44.641888 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6363cc27-2fc3-4bcc-91ff-285be3ae7fd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:35:44.643563 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6363cc27-2fc3-4bcc-91ff-285be3ae7fd9 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:35:44.654520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1133/2240] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:35:44.658333 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6363cc27-2fc3-4bcc-91ff-285be3ae7fd9 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:35:44.665278 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e20718-5677-485b-bc06-7be7ffefe012 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84b02e7351914cabb67462454b328d05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:35:44.665895 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e20718-5677-485b-bc06-7be7ffefe012 service placement] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:44.666210 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e20718-5677-485b-bc06-7be7ffefe012 service placement] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:44.666488 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e20718-5677-485b-bc06-7be7ffefe012 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:44.668820 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-93e20718-5677-485b-bc06-7be7ffefe012 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:35:44.670230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6363cc27-2fc3-4bcc-91ff-285be3ae7fd9 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:35:44.675510 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c1fa6-af72-474b-89b8-b02bb37e6728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:35:44.676701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c1fa6-af72-474b-89b8-b02bb37e6728 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:44.677095 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c1fa6-af72-474b-89b8-b02bb37e6728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:44.677562 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c1fa6-af72-474b-89b8-b02bb37e6728 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:44.678377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c1fa6-af72-474b-89b8-b02bb37e6728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:35:44.679957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c1fa6-af72-474b-89b8-b02bb37e6728 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:35:44.680620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e20718-5677-485b-bc06-7be7ffefe012 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:35:44.682225 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e20718-5677-485b-bc06-7be7ffefe012 service placement] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:35:44.691589 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1134/2241] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:35:44.693081 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1108/2242] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:35:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:35:44.695389 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c1fa6-af72-474b-89b8-b02bb37e6728 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:35:44.705426 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c1fa6-af72-474b-89b8-b02bb37e6728 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:35:44.710265 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a350b-9756-4e19-bf1a-679657cabae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:35:44.710982 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a350b-9756-4e19-bf1a-679657cabae6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:44.711220 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a350b-9756-4e19-bf1a-679657cabae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:44.711640 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a350b-9756-4e19-bf1a-679657cabae6 admin admin] PATH_INFO: `/v3/projects/29154ac37669476cbd7b159f41d1ec07/users/cb9988fa3e29499fb2f5b2fe8f11089b/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:44.728252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6a350b-9756-4e19-bf1a-679657cabae6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29154ac37669476cbd7b159f41d1ec07, user_id=cb9988fa3e29499fb2f5b2fe8f11089b, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:35:44.731528 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6a350b-9756-4e19-bf1a-679657cabae6 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:35:44.745234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1135/2243] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:35:44 2022] PUT /identity/v3/projects/29154ac37669476cbd7b159f41d1ec07/users/cb9988fa3e29499fb2f5b2fe8f11089b/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:35:44.755479 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4571ad3-3cb4-4397-88ea-05dcca47871e None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:44.756006 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4571ad3-3cb4-4397-88ea-05dcca47871e None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:44.756006 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4571ad3-3cb4-4397-88ea-05dcca47871e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:44.784495 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-a4571ad3-3cb4-4397-88ea-05dcca47871e None None] MFA Rules not processed for user `cb9988fa3e29499fb2f5b2fe8f11089b`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:35:44.785710 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-a4571ad3-3cb4-4397-88ea-05dcca47871e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:35:44.857009 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1109/2244] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:35:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:35:44.868409 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a350b-9756-4e19-bf1a-679657cabae6 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:35:44.870350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-da6a350b-9756-4e19-bf1a-679657cabae6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:35:44.928738 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a350b-9756-4e19-bf1a-679657cabae6 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:35:44.932599 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9355a6-2ddd-4c70-a711-7f29518e6869 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:35:44.933203 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9355a6-2ddd-4c70-a711-7f29518e6869 service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:44.933491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9355a6-2ddd-4c70-a711-7f29518e6869 service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:44.933757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9355a6-2ddd-4c70-a711-7f29518e6869 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:44.942111 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9355a6-2ddd-4c70-a711-7f29518e6869 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:35:44.943515 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9355a6-2ddd-4c70-a711-7f29518e6869 service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:35:44.953265 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1136/2245] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:35:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:35:54.854048 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=90 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:35:54.948484 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=87,keystone:INSERT=5 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:35:58.836615 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21c7e1-b2fb-4f94-aec4-e749057c7921 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:58.837584 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21c7e1-b2fb-4f94-aec4-e749057c7921 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:58.837584 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21c7e1-b2fb-4f94-aec4-e749057c7921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:58.857410 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-8f21c7e1-b2fb-4f94-aec4-e749057c7921 None None] MFA Rules not processed for user `9158557f1a2947e2815dc8d586d8b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:35:58.858734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-8f21c7e1-b2fb-4f94-aec4-e749057c7921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:35:58.918493 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1110/2246] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Sep 26 10:35:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:35:58.933003 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9355a6-2ddd-4c70-a711-7f29518e6869 service cinder] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:35:58.935412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-dc9355a6-2ddd-4c70-a711-7f29518e6869 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:35:59.016931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9355a6-2ddd-4c70-a711-7f29518e6869 service cinder] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:35:59.022521 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47579650-3f51-4ca0-8a3e-97edcb17656e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e6be34fc93a4551a83e5b8380188a6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:35:59.023152 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47579650-3f51-4ca0-8a3e-97edcb17656e service swift] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:35:59.023431 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47579650-3f51-4ca0-8a3e-97edcb17656e service swift] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:35:59.023702 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47579650-3f51-4ca0-8a3e-97edcb17656e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:35:59.032330 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47579650-3f51-4ca0-8a3e-97edcb17656e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:35:59.033927 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47579650-3f51-4ca0-8a3e-97edcb17656e service swift] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:35:59.042445 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1137/2247] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 10:35:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 10:36:02.956475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f21c7e1-b2fb-4f94-aec4-e749057c7921 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:36:02.958276 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-8f21c7e1-b2fb-4f94-aec4-e749057c7921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:36:03.028517 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f21c7e1-b2fb-4f94-aec4-e749057c7921 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:36:03.032191 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06692483-4f97-416e-b24e-c898326a0cc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:36:03.032799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06692483-4f97-416e-b24e-c898326a0cc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:03.033070 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06692483-4f97-416e-b24e-c898326a0cc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:03.033338 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06692483-4f97-416e-b24e-c898326a0cc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:03.036158 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-06692483-4f97-416e-b24e-c898326a0cc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:36:03.118112 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06692483-4f97-416e-b24e-c898326a0cc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:36:03.119702 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06692483-4f97-416e-b24e-c898326a0cc6 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:36:03.140380 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1111/2248] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:36:02 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:36:09.039445 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=32 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:36:13.130969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=75 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:36:13.678156 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae0f53-cdcf-4492-9874-8002c6950343 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:13.678722 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae0f53-cdcf-4492-9874-8002c6950343 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:13.678722 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae0f53-cdcf-4492-9874-8002c6950343 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:13.680375 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1138/2249] 10.222.0.49 () {54 vars in 921 bytes} [Mon Sep 26 10:36:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:36:13.686472 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc3f04-7d49-4af4-b707-4c618c3cb576 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:13.686778 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc3f04-7d49-4af4-b707-4c618c3cb576 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:13.687049 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc3f04-7d49-4af4-b707-4c618c3cb576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:13.708011 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-e1fc3f04-7d49-4af4-b707-4c618c3cb576 None None] MFA Rules not processed for user `cee41014d5d043888c6b399710bccab4`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:36:13.709224 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-e1fc3f04-7d49-4af4-b707-4c618c3cb576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:36:13.794656 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1112/2250] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Sep 26 10:36:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:36:13.814510 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae0f53-cdcf-4492-9874-8002c6950343 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:36:13.816337 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-c6ae0f53-cdcf-4492-9874-8002c6950343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:36:13.880963 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae0f53-cdcf-4492-9874-8002c6950343 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:36:13.884495 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1a328-5b1d-4b78-afcb-b791059e934c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41b242fbe2504f8cac26ce4ff46f5390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:36:13.885098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1a328-5b1d-4b78-afcb-b791059e934c service glance] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:13.885377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1a328-5b1d-4b78-afcb-b791059e934c service glance] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:13.885649 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1a328-5b1d-4b78-afcb-b791059e934c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:13.895646 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1a328-5b1d-4b78-afcb-b791059e934c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:36:13.897058 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1a328-5b1d-4b78-afcb-b791059e934c service glance] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:36:13.908304 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1139/2251] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:36:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:36:13.975465 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a030b80-f7fe-4cbd-8348-cc3a74bacb8c None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:13.975699 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a030b80-f7fe-4cbd-8348-cc3a74bacb8c None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:13.975971 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a030b80-f7fe-4cbd-8348-cc3a74bacb8c None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:13.977561 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1113/2252] 10.222.0.49 () {54 vars in 921 bytes} [Mon Sep 26 10:36:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:36:13.982246 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ada86-4a1d-4742-82a7-236383994104 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:13.982525 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ada86-4a1d-4742-82a7-236383994104 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:13.982843 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ada86-4a1d-4742-82a7-236383994104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:14.002683 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-9e5ada86-4a1d-4742-82a7-236383994104 None None] MFA Rules not processed for user `f841bf31c43b4333a3d834d38b5197e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:36:14.003877 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-9e5ada86-4a1d-4742-82a7-236383994104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:36:14.010533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1140/2253] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Sep 26 10:36:13 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:36:14.022471 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a030b80-f7fe-4cbd-8348-cc3a74bacb8c None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:36:14.024339 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-6a030b80-f7fe-4cbd-8348-cc3a74bacb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:36:14.039545 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a030b80-f7fe-4cbd-8348-cc3a74bacb8c None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:36:14.043042 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9031b292-f8c9-4fa1-824e-c5e6d273319b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:36:14.043625 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031b292-f8c9-4fa1-824e-c5e6d273319b service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:14.043898 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031b292-f8c9-4fa1-824e-c5e6d273319b service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:14.044165 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031b292-f8c9-4fa1-824e-c5e6d273319b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:14.050990 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9031b292-f8c9-4fa1-824e-c5e6d273319b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:36:14.052439 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9031b292-f8c9-4fa1-824e-c5e6d273319b service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:36:14.059988 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1114/2254] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:36:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:36:14.141751 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d35cbf-2729-47b5-826f-2a1b37cbe728 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:14.142067 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d35cbf-2729-47b5-826f-2a1b37cbe728 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:14.142349 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d35cbf-2729-47b5-826f-2a1b37cbe728 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:14.143695 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1141/2255] 10.222.0.49 () {54 vars in 921 bytes} [Mon Sep 26 10:36:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:36:14.148578 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7f5d2-5b87-450f-bb6e-385d17bc8bb1 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:14.148997 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7f5d2-5b87-450f-bb6e-385d17bc8bb1 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:14.149157 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7f5d2-5b87-450f-bb6e-385d17bc8bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:14.169518 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-01c7f5d2-5b87-450f-bb6e-385d17bc8bb1 None None] MFA Rules not processed for user `cee41014d5d043888c6b399710bccab4`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:36:14.170760 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-01c7f5d2-5b87-450f-bb6e-385d17bc8bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:36:14.180044 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1115/2256] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Sep 26 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:36:14.190347 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d35cbf-2729-47b5-826f-2a1b37cbe728 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:36:14.201076 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d35cbf-2729-47b5-826f-2a1b37cbe728 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:36:14.205375 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415a2c7-05da-405a-87b1-e0ac35d065d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:36:14.205976 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415a2c7-05da-405a-87b1-e0ac35d065d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:14.206299 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415a2c7-05da-405a-87b1-e0ac35d065d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:14.206574 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415a2c7-05da-405a-87b1-e0ac35d065d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:14.216801 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c415a2c7-05da-405a-87b1-e0ac35d065d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:36:14.218846 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c415a2c7-05da-405a-87b1-e0ac35d065d4 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:36:14.229780 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1142/2257] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:36:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:36:16.929792 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2342d-0240-42e3-b47b-629b67dfb9a7 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:16.930289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2342d-0240-42e3-b47b-629b67dfb9a7 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:16.930422 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2342d-0240-42e3-b47b-629b67dfb9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:16.949309 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-5af2342d-0240-42e3-b47b-629b67dfb9a7 None None] MFA Rules not processed for user `9158557f1a2947e2815dc8d586d8b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:36:16.950535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-5af2342d-0240-42e3-b47b-629b67dfb9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:36:16.957567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1116/2258] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Sep 26 10:36:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:36:16.979128 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415a2c7-05da-405a-87b1-e0ac35d065d4 service neutron] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:36:16.990927 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415a2c7-05da-405a-87b1-e0ac35d065d4 service neutron] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:36:16.995793 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5002d06f-310f-455f-9a0f-4d0da48daa64 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e6be34fc93a4551a83e5b8380188a6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:36:16.996378 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002d06f-310f-455f-9a0f-4d0da48daa64 service swift] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:16.996652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002d06f-310f-455f-9a0f-4d0da48daa64 service swift] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:16.996922 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002d06f-310f-455f-9a0f-4d0da48daa64 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:17.005010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5002d06f-310f-455f-9a0f-4d0da48daa64 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:36:17.006637 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5002d06f-310f-455f-9a0f-4d0da48daa64 service swift] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:36:17.015712 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1143/2259] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 10:36:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 10:36:22.821885 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2342d-0240-42e3-b47b-629b67dfb9a7 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:36:22.888696 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2342d-0240-42e3-b47b-629b67dfb9a7 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:36:22.893733 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e518154-fc31-4547-9fb3-842100d1007f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:36:22.894503 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e518154-fc31-4547-9fb3-842100d1007f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:22.894809 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e518154-fc31-4547-9fb3-842100d1007f admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:22.895085 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e518154-fc31-4547-9fb3-842100d1007f admin admin] PATH_INFO: `/v3/users/cb9988fa3e29499fb2f5b2fe8f11089b` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:22.896228 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e518154-fc31-4547-9fb3-842100d1007f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb9988fa3e29499fb2f5b2fe8f11089b)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:36:22.898180 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e518154-fc31-4547-9fb3-842100d1007f admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:36:22.953872 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-3e518154-fc31-4547-9fb3-842100d1007f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb9988fa3e29499fb2f5b2fe8f11089b'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:36:22.954556 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-3e518154-fc31-4547-9fb3-842100d1007f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb9988fa3e29499fb2f5b2fe8f11089b'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:36:22.955151 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-3e518154-fc31-4547-9fb3-842100d1007f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb9988fa3e29499fb2f5b2fe8f11089b'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:36:22.962064 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-3e518154-fc31-4547-9fb3-842100d1007f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb9988fa3e29499fb2f5b2fe8f11089b'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:36:22.970330 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:36:22.976091 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-3e518154-fc31-4547-9fb3-842100d1007f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb9988fa3e29499fb2f5b2fe8f11089b'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:36:22.990456 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1117/2260] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:36:22 2022] DELETE /identity/v3/users/cb9988fa3e29499fb2f5b2fe8f11089b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:36:22.995017 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea82d4a-8fbd-46a1-81f2-384e0cab01c7 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:22.995547 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea82d4a-8fbd-46a1-81f2-384e0cab01c7 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:22.995547 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea82d4a-8fbd-46a1-81f2-384e0cab01c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:23.021286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-eea82d4a-8fbd-46a1-81f2-384e0cab01c7 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:36:23.022541 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-eea82d4a-8fbd-46a1-81f2-384e0cab01c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:36:23.085131 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1144/2261] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:36:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:36:23.095269 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e518154-fc31-4547-9fb3-842100d1007f admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:36:23.096668 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-3e518154-fc31-4547-9fb3-842100d1007f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:36:23.156813 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e518154-fc31-4547-9fb3-842100d1007f admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:36:23.160808 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26470ff-6724-4413-9e94-bd8a46ee4d72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:36:23.161516 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26470ff-6724-4413-9e94-bd8a46ee4d72 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:23.161797 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26470ff-6724-4413-9e94-bd8a46ee4d72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:23.162101 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26470ff-6724-4413-9e94-bd8a46ee4d72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:23.172494 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26470ff-6724-4413-9e94-bd8a46ee4d72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:36:23.173937 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26470ff-6724-4413-9e94-bd8a46ee4d72 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:36:23.184639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1118/2262] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:36:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:36:23.599667 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea82d4a-8fbd-46a1-81f2-384e0cab01c7 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:36:23.601103 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-eea82d4a-8fbd-46a1-81f2-384e0cab01c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:36:23.613428 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea82d4a-8fbd-46a1-81f2-384e0cab01c7 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:36:23.619753 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d712b146-d40d-427c-b480-cb03c9126a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:36:23.620373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712b146-d40d-427c-b480-cb03c9126a9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:23.620679 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712b146-d40d-427c-b480-cb03c9126a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:23.620917 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712b146-d40d-427c-b480-cb03c9126a9d admin admin] PATH_INFO: `/v3/projects/29154ac37669476cbd7b159f41d1ec07` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:23.622058 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d712b146-d40d-427c-b480-cb03c9126a9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=29154ac37669476cbd7b159f41d1ec07)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:36:23.637730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d712b146-d40d-427c-b480-cb03c9126a9d admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:36:23.698941 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d712b146-d40d-427c-b480-cb03c9126a9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29154ac37669476cbd7b159f41d1ec07', 'request_id': 'req-d712b146-d40d-427c-b480-cb03c9126a9d'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:36:23.706638 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d712b146-d40d-427c-b480-cb03c9126a9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29154ac37669476cbd7b159f41d1ec07', 'request_id': 'req-d712b146-d40d-427c-b480-cb03c9126a9d'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:36:23.715871 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d712b146-d40d-427c-b480-cb03c9126a9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29154ac37669476cbd7b159f41d1ec07', 'request_id': 'req-d712b146-d40d-427c-b480-cb03c9126a9d'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:36:23.736087 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:36:23.744795 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d712b146-d40d-427c-b480-cb03c9126a9d admin admin] The token cache is being invalidate because project 29154ac37669476cbd7b159f41d1ec07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:36:23.745216 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d712b146-d40d-427c-b480-cb03c9126a9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:36:23.746552 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d712b146-d40d-427c-b480-cb03c9126a9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:36:23.749947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1145/2263] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:36:23 2022] DELETE /identity/v3/projects/29154ac37669476cbd7b159f41d1ec07 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:36:23.755701 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519fa068-66fe-4cf3-9f42-49ae307a5a36 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:23.756055 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519fa068-66fe-4cf3-9f42-49ae307a5a36 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:23.756408 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519fa068-66fe-4cf3-9f42-49ae307a5a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:23.780738 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-519fa068-66fe-4cf3-9f42-49ae307a5a36 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:36:23.782793 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-519fa068-66fe-4cf3-9f42-49ae307a5a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:36:23.860357 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1119/2264] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:36:23.868335 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189d6c0-e276-4991-b2ae-866bd8a8ac7e None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:23.868615 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189d6c0-e276-4991-b2ae-866bd8a8ac7e None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:23.868889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189d6c0-e276-4991-b2ae-866bd8a8ac7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:23.890131 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-7189d6c0-e276-4991-b2ae-866bd8a8ac7e None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:36:23.891338 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-7189d6c0-e276-4991-b2ae-866bd8a8ac7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:36:23.899730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1146/2265] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:36:23.904139 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519fa068-66fe-4cf3-9f42-49ae307a5a36 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:36:23.914322 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519fa068-66fe-4cf3-9f42-49ae307a5a36 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:36:23.918821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1404e-5214-4a45-8909-dd09216788b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:36:23.919411 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1404e-5214-4a45-8909-dd09216788b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:23.919687 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1404e-5214-4a45-8909-dd09216788b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:23.919959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1404e-5214-4a45-8909-dd09216788b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:23.921267 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb1404e-5214-4a45-8909-dd09216788b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:36:23.921684 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb1404e-5214-4a45-8909-dd09216788b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:36:23.923039 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb1404e-5214-4a45-8909-dd09216788b2 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:36:23.942504 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1120/2266] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:36:23.943425 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7189d6c0-e276-4991-b2ae-866bd8a8ac7e None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:36:23.953997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7189d6c0-e276-4991-b2ae-866bd8a8ac7e None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:36:23.958943 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70ae15-4d0f-4d29-b2d5-043dbc0ee1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:36:23.959530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70ae15-4d0f-4d29-b2d5-043dbc0ee1b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:23.959805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70ae15-4d0f-4d29-b2d5-043dbc0ee1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:23.960207 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70ae15-4d0f-4d29-b2d5-043dbc0ee1b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:23.961283 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb70ae15-4d0f-4d29-b2d5-043dbc0ee1b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:36:23.975875 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb70ae15-4d0f-4d29-b2d5-043dbc0ee1b5 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:36:24.002411 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1147/2267] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:36:23 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:36:24.003254 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1404e-5214-4a45-8909-dd09216788b2 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:36:24.013582 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1404e-5214-4a45-8909-dd09216788b2 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:36:24.084871 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c35b8-ee4c-4060-b048-f33f11d36a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:36:24.085599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c35b8-ee4c-4060-b048-f33f11d36a71 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:24.085945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c35b8-ee4c-4060-b048-f33f11d36a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:24.086340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c35b8-ee4c-4060-b048-f33f11d36a71 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:24.087050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c35b8-ee4c-4060-b048-f33f11d36a71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:36:24.089032 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c35b8-ee4c-4060-b048-f33f11d36a71 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:36:24.115778 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1121/2268] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:36:24 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:36:24.119842 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70ae15-4d0f-4d29-b2d5-043dbc0ee1b5 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:36:24.132808 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70ae15-4d0f-4d29-b2d5-043dbc0ee1b5 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:36:24.138589 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29521a-e395-435c-8d6b-8126d854c131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:36:24.139448 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29521a-e395-435c-8d6b-8126d854c131 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:24.139723 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29521a-e395-435c-8d6b-8126d854c131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:24.140503 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29521a-e395-435c-8d6b-8126d854c131 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:24.141239 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a29521a-e395-435c-8d6b-8126d854c131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:36:24.142810 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a29521a-e395-435c-8d6b-8126d854c131 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:36:24.165807 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1148/2269] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:36:24.167288 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c35b8-ee4c-4060-b048-f33f11d36a71 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:36:24.176029 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:36:24.180900 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c35b8-ee4c-4060-b048-f33f11d36a71 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:36:24.202811 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c7170-674f-4a75-912e-325ce90e1d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:36:24.203505 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c7170-674f-4a75-912e-325ce90e1d7e admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:24.203848 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c7170-674f-4a75-912e-325ce90e1d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:24.204187 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c7170-674f-4a75-912e-325ce90e1d7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:24.204918 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c7170-674f-4a75-912e-325ce90e1d7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:36:24.206858 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c7170-674f-4a75-912e-325ce90e1d7e admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:36:24.240106 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1122/2270] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:36:24.244605 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29521a-e395-435c-8d6b-8126d854c131 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:36:24.258181 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29521a-e395-435c-8d6b-8126d854c131 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:36:24.263317 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e578c76-babc-4936-aedf-307c86b57348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:36:24.263980 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e578c76-babc-4936-aedf-307c86b57348 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:24.264286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e578c76-babc-4936-aedf-307c86b57348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:24.264558 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e578c76-babc-4936-aedf-307c86b57348 admin admin] PATH_INFO: `/v3/projects/16a8a5fc67624a9eaa29eeec94dfe2c5/users/554d4e895fc542cfacb42990cc2e3510/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:24.292113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e578c76-babc-4936-aedf-307c86b57348 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16a8a5fc67624a9eaa29eeec94dfe2c5, user_id=554d4e895fc542cfacb42990cc2e3510, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:36:24.298054 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e578c76-babc-4936-aedf-307c86b57348 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:36:24.324317 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1149/2271] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:36:24 2022] PUT /identity/v3/projects/16a8a5fc67624a9eaa29eeec94dfe2c5/users/554d4e895fc542cfacb42990cc2e3510/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:36:24.336184 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fdf59-93b5-4f3e-9592-51d231a95e83 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:24.336532 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fdf59-93b5-4f3e-9592-51d231a95e83 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:24.342856 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fdf59-93b5-4f3e-9592-51d231a95e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:24.405843 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-140fdf59-93b5-4f3e-9592-51d231a95e83 None None] MFA Rules not processed for user `554d4e895fc542cfacb42990cc2e3510`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:36:24.407125 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-140fdf59-93b5-4f3e-9592-51d231a95e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:36:24.510084 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1123/2272] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:36:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:36:24.522114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e578c76-babc-4936-aedf-307c86b57348 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:36:24.524420 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-3e578c76-babc-4936-aedf-307c86b57348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:36:24.623868 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e578c76-babc-4936-aedf-307c86b57348 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:36:24.628917 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4edb0-1909-4ed5-a505-92731a514308 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:36:24.630474 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4edb0-1909-4ed5-a505-92731a514308 service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:24.630761 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4edb0-1909-4ed5-a505-92731a514308 service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:24.631605 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4edb0-1909-4ed5-a505-92731a514308 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:24.669623 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef4edb0-1909-4ed5-a505-92731a514308 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:36:24.671841 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef4edb0-1909-4ed5-a505-92731a514308 service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:36:24.690471 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1150/2273] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:36:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:36:24.720314 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140fdf59-93b5-4f3e-9592-51d231a95e83 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:36:24.825222 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140fdf59-93b5-4f3e-9592-51d231a95e83 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:36:24.831046 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92fbb4-d8a8-4134-a2e4-1cfc95617194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:36:24.831882 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92fbb4-d8a8-4134-a2e4-1cfc95617194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:24.832239 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92fbb4-d8a8-4134-a2e4-1cfc95617194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:24.832552 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92fbb4-d8a8-4134-a2e4-1cfc95617194 admin admin] PATH_INFO: `/v3/users/554d4e895fc542cfacb42990cc2e3510` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:24.833753 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd92fbb4-d8a8-4134-a2e4-1cfc95617194 admin admin] RBAC: Authorizing `identity:delete_user(user_id=554d4e895fc542cfacb42990cc2e3510)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:36:24.835926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd92fbb4-d8a8-4134-a2e4-1cfc95617194 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:36:24.886979 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-dd92fbb4-d8a8-4134-a2e4-1cfc95617194 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '554d4e895fc542cfacb42990cc2e3510'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:36:24.888083 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-dd92fbb4-d8a8-4134-a2e4-1cfc95617194 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '554d4e895fc542cfacb42990cc2e3510'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:36:24.888699 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-dd92fbb4-d8a8-4134-a2e4-1cfc95617194 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '554d4e895fc542cfacb42990cc2e3510'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:36:24.896861 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-dd92fbb4-d8a8-4134-a2e4-1cfc95617194 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '554d4e895fc542cfacb42990cc2e3510'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:36:24.914882 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-dd92fbb4-d8a8-4134-a2e4-1cfc95617194 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '554d4e895fc542cfacb42990cc2e3510'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:36:24.929059 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1124/2274] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:36:24 2022] DELETE /identity/v3/users/554d4e895fc542cfacb42990cc2e3510 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:36:24.934377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdaf4aa-e561-4492-bf3d-0bfed7e9cb7f None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:24.934787 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdaf4aa-e561-4492-bf3d-0bfed7e9cb7f None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:24.935130 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdaf4aa-e561-4492-bf3d-0bfed7e9cb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:24.955503 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-2cdaf4aa-e561-4492-bf3d-0bfed7e9cb7f None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:36:24.956852 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-2cdaf4aa-e561-4492-bf3d-0bfed7e9cb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:36:25.032884 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1151/2275] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:36:25.042644 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92fbb4-d8a8-4134-a2e4-1cfc95617194 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:36:25.044070 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-dd92fbb4-d8a8-4134-a2e4-1cfc95617194 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:36:25.049655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:36:25.114859 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92fbb4-d8a8-4134-a2e4-1cfc95617194 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:36:25.118542 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4fbf3-32a5-467a-aabc-db408d795b10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:36:25.119139 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4fbf3-32a5-467a-aabc-db408d795b10 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:25.119414 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4fbf3-32a5-467a-aabc-db408d795b10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:25.119681 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4fbf3-32a5-467a-aabc-db408d795b10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:25.129458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4fbf3-32a5-467a-aabc-db408d795b10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:36:25.130931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4fbf3-32a5-467a-aabc-db408d795b10 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:36:25.142238 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1125/2276] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:36:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:36:25.463332 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdaf4aa-e561-4492-bf3d-0bfed7e9cb7f None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:36:25.465020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-2cdaf4aa-e561-4492-bf3d-0bfed7e9cb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:36:25.491050 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdaf4aa-e561-4492-bf3d-0bfed7e9cb7f None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:36:25.496977 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1239a5-06cd-499a-bc2b-4dd28767131b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:36:25.497712 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1239a5-06cd-499a-bc2b-4dd28767131b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:25.498096 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1239a5-06cd-499a-bc2b-4dd28767131b admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:25.498449 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1239a5-06cd-499a-bc2b-4dd28767131b admin admin] PATH_INFO: `/v3/projects/16a8a5fc67624a9eaa29eeec94dfe2c5` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:25.500924 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1239a5-06cd-499a-bc2b-4dd28767131b admin admin] RBAC: Authorizing `identity:delete_project(project_id=16a8a5fc67624a9eaa29eeec94dfe2c5)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:36:25.502874 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1239a5-06cd-499a-bc2b-4dd28767131b admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:36:25.549794 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:36:25.568253 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-1b1239a5-06cd-499a-bc2b-4dd28767131b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16a8a5fc67624a9eaa29eeec94dfe2c5', 'request_id': 'req-1b1239a5-06cd-499a-bc2b-4dd28767131b'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:36:25.574926 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-1b1239a5-06cd-499a-bc2b-4dd28767131b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16a8a5fc67624a9eaa29eeec94dfe2c5', 'request_id': 'req-1b1239a5-06cd-499a-bc2b-4dd28767131b'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:36:25.585336 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-1b1239a5-06cd-499a-bc2b-4dd28767131b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16a8a5fc67624a9eaa29eeec94dfe2c5', 'request_id': 'req-1b1239a5-06cd-499a-bc2b-4dd28767131b'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:36:25.598159 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-1b1239a5-06cd-499a-bc2b-4dd28767131b admin admin] The token cache is being invalidate because project 16a8a5fc67624a9eaa29eeec94dfe2c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:36:25.598447 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-1b1239a5-06cd-499a-bc2b-4dd28767131b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:36:25.599114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-1b1239a5-06cd-499a-bc2b-4dd28767131b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:36:25.600784 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1152/2277] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:36:25 2022] DELETE /identity/v3/projects/16a8a5fc67624a9eaa29eeec94dfe2c5 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:36:35.136583 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=25 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:36:35.594673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=14,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:36:41.835505 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b50d21-8c24-486a-8627-a786524329ac None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:41.835993 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b50d21-8c24-486a-8627-a786524329ac None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:41.835993 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b50d21-8c24-486a-8627-a786524329ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:41.858250 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-13b50d21-8c24-486a-8627-a786524329ac None None] MFA Rules not processed for user `9158557f1a2947e2815dc8d586d8b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:36:41.859497 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-13b50d21-8c24-486a-8627-a786524329ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:36:41.923918 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1126/2278] 10.222.0.49 () {58 vars in 967 bytes} [Mon Sep 26 10:36:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:36:41.937380 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1239a5-06cd-499a-bc2b-4dd28767131b admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:36:41.938946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-1b1239a5-06cd-499a-bc2b-4dd28767131b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:36:42.012174 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1239a5-06cd-499a-bc2b-4dd28767131b admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:36:42.017588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72771f68-992e-4492-88e8-1830de2aa805 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e6be34fc93a4551a83e5b8380188a6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:36:42.018288 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72771f68-992e-4492-88e8-1830de2aa805 service swift] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:42.018570 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72771f68-992e-4492-88e8-1830de2aa805 service swift] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:42.018842 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72771f68-992e-4492-88e8-1830de2aa805 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:42.027561 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72771f68-992e-4492-88e8-1830de2aa805 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:36:42.029279 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72771f68-992e-4492-88e8-1830de2aa805 service swift] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:36:42.038360 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1153/2279] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 10:36:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 10:36:45.850481 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b874b9-f25d-4f32-9724-99bd71c3b6e3 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:36:45.851348 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b874b9-f25d-4f32-9724-99bd71c3b6e3 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:36:45.851795 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b874b9-f25d-4f32-9724-99bd71c3b6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:36:45.875412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-c5b874b9-f25d-4f32-9724-99bd71c3b6e3 None None] MFA Rules not processed for user `b486a094550541729c19c8713a648243`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:36:45.876638 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-c5b874b9-f25d-4f32-9724-99bd71c3b6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:36:45.934577 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1127/2280] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:36:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:36:52.034715 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=32 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:36:55.928030 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=50 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:37:01.331060 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae49b4-bc97-4da3-9e3e-3cc0f19508fd None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:37:01.331580 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae49b4-bc97-4da3-9e3e-3cc0f19508fd None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:37:01.331580 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae49b4-bc97-4da3-9e3e-3cc0f19508fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:37:01.351008 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-17ae49b4-bc97-4da3-9e3e-3cc0f19508fd None None] MFA Rules not processed for user `5085a918c6b74488b3dd70c1fb257381`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:37:01.352190 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-17ae49b4-bc97-4da3-9e3e-3cc0f19508fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:37:01.406225 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1154/2281] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:37:01 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:37:11.398947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=25 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:37:30.670429 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c034d-755a-4c09-81b4-0a3b6ad1255f None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:37:30.670959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c034d-755a-4c09-81b4-0a3b6ad1255f None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:37:30.670959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c034d-755a-4c09-81b4-0a3b6ad1255f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:37:30.690728 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-a10c034d-755a-4c09-81b4-0a3b6ad1255f None None] MFA Rules not processed for user `8c8ebfdad6ab437eb7dda111fee84205`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:37:30.691943 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-a10c034d-755a-4c09-81b4-0a3b6ad1255f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:37:30.749307 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1128/2282] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:37:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:37:32.492743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae49b4-bc97-4da3-9e3e-3cc0f19508fd None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:37:32.494416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-17ae49b4-bc97-4da3-9e3e-3cc0f19508fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:37:32.560990 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae49b4-bc97-4da3-9e3e-3cc0f19508fd None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:37:32.565190 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f3435-022e-4937-9728-20f003f41027 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:37:32.565976 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f3435-022e-4937-9728-20f003f41027 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:37:32.566608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f3435-022e-4937-9728-20f003f41027 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:37:32.566837 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f3435-022e-4937-9728-20f003f41027 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:37:32.569729 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-ee0f3435-022e-4937-9728-20f003f41027 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:37:32.655939 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0f3435-022e-4937-9728-20f003f41027 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:37:32.657471 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0f3435-022e-4937-9728-20f003f41027 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:37:32.671114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1155/2283] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:37:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:37:32.899564 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c034d-755a-4c09-81b4-0a3b6ad1255f None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:37:32.909348 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c034d-755a-4c09-81b4-0a3b6ad1255f None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:37:32.912981 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18b7349-7085-4242-a632-8d0b1428e788 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:37:32.913582 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b7349-7085-4242-a632-8d0b1428e788 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:37:32.913857 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b7349-7085-4242-a632-8d0b1428e788 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:37:32.914164 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b7349-7085-4242-a632-8d0b1428e788 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:37:32.915905 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-e18b7349-7085-4242-a632-8d0b1428e788 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:37:32.931222 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18b7349-7085-4242-a632-8d0b1428e788 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:37:32.933078 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18b7349-7085-4242-a632-8d0b1428e788 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:37:32.946673 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1129/2284] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:37:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:37:36.538316 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836a180-f634-417f-9a07-83140a4e0d0c None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:37:36.538865 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836a180-f634-417f-9a07-83140a4e0d0c None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:37:36.538865 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836a180-f634-417f-9a07-83140a4e0d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:37:36.557722 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-1836a180-f634-417f-9a07-83140a4e0d0c None None] MFA Rules not processed for user `cdcfab461e534465a906c0eef201e423`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:37:36.558973 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-1836a180-f634-417f-9a07-83140a4e0d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:37:36.615358 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1156/2285] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:37:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:37:42.941444 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=31 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:37:46.608554 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=79 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:37:48.969988 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18b7349-7085-4242-a632-8d0b1428e788 service neutron] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:37:48.971580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-e18b7349-7085-4242-a632-8d0b1428e788 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:37:48.985446 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18b7349-7085-4242-a632-8d0b1428e788 service neutron] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:37:48.990891 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16db082-ecfa-436f-8cd6-1431bc7ad7d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:37:48.991481 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16db082-ecfa-436f-8cd6-1431bc7ad7d3 service nova] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:37:48.991765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16db082-ecfa-436f-8cd6-1431bc7ad7d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:37:48.992037 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16db082-ecfa-436f-8cd6-1431bc7ad7d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:37:48.993729 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-c16db082-ecfa-436f-8cd6-1431bc7ad7d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:37:49.003833 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16db082-ecfa-436f-8cd6-1431bc7ad7d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:37:49.005942 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16db082-ecfa-436f-8cd6-1431bc7ad7d3 service nova] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:37:49.014838 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1130/2286] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:37:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:37:49.536191 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836a180-f634-417f-9a07-83140a4e0d0c None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:37:49.549132 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836a180-f634-417f-9a07-83140a4e0d0c None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:37:49.554600 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81dab96-3ba5-4134-ae8f-10ba10ab231a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:37:49.555202 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81dab96-3ba5-4134-ae8f-10ba10ab231a service nova] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:37:49.555479 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81dab96-3ba5-4134-ae8f-10ba10ab231a service nova] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:37:49.555746 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81dab96-3ba5-4134-ae8f-10ba10ab231a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:37:49.557472 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-b81dab96-3ba5-4134-ae8f-10ba10ab231a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:37:49.568405 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81dab96-3ba5-4134-ae8f-10ba10ab231a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:37:49.569973 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81dab96-3ba5-4134-ae8f-10ba10ab231a service nova] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:37:49.579321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1157/2287] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:37:49 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:37:53.670259 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16db082-ecfa-436f-8cd6-1431bc7ad7d3 service nova] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:37:53.681508 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16db082-ecfa-436f-8cd6-1431bc7ad7d3 service nova] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:37:53.688365 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b2c49-4b74-433b-831d-def1c531ec46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:37:53.689168 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b2c49-4b74-433b-831d-def1c531ec46 service nova] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:37:53.689447 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b2c49-4b74-433b-831d-def1c531ec46 service nova] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:37:53.689715 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b2c49-4b74-433b-831d-def1c531ec46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:37:53.691874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-bd7b2c49-4b74-433b-831d-def1c531ec46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:37:53.702693 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b2c49-4b74-433b-831d-def1c531ec46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:37:53.704318 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b2c49-4b74-433b-831d-def1c531ec46 service nova] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:37:53.714147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1131/2288] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:37:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:37:59.575520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=6 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:02.363224 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81dab96-3ba5-4134-ae8f-10ba10ab231a service nova] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:02.374063 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81dab96-3ba5-4134-ae8f-10ba10ab231a service nova] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:02.378125 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee9060-0865-435d-8bc2-e5722dbdb700 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:02.378971 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee9060-0865-435d-8bc2-e5722dbdb700 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:02.379436 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee9060-0865-435d-8bc2-e5722dbdb700 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:02.379946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee9060-0865-435d-8bc2-e5722dbdb700 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:02.381947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-c6ee9060-0865-435d-8bc2-e5722dbdb700 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:02.392559 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ee9060-0865-435d-8bc2-e5722dbdb700 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:02.394075 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ee9060-0865-435d-8bc2-e5722dbdb700 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:02.403227 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1158/2289] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:38:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:38:03.709857 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=12 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:04.272551 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f808e-3c8d-4ff3-afb0-ec2760767e29 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:04.272837 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f808e-3c8d-4ff3-afb0-ec2760767e29 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:04.273203 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f808e-3c8d-4ff3-afb0-ec2760767e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:04.292437 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-0e7f808e-3c8d-4ff3-afb0-ec2760767e29 None None] MFA Rules not processed for user `b486a094550541729c19c8713a648243`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:38:04.293634 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-0e7f808e-3c8d-4ff3-afb0-ec2760767e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:04.300768 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1132/2290] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:38:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:38:04.684849 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee9060-0865-435d-8bc2-e5722dbdb700 service neutron] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:04.693684 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee9060-0865-435d-8bc2-e5722dbdb700 service neutron] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:04.697251 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad3bd8-d1fc-41a9-bf21-d2ad65ad51f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:04.697835 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad3bd8-d1fc-41a9-bf21-d2ad65ad51f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:04.698138 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad3bd8-d1fc-41a9-bf21-d2ad65ad51f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:04.698408 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad3bd8-d1fc-41a9-bf21-d2ad65ad51f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:04.700081 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-dcad3bd8-d1fc-41a9-bf21-d2ad65ad51f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:04.709835 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcad3bd8-d1fc-41a9-bf21-d2ad65ad51f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:04.711267 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcad3bd8-d1fc-41a9-bf21-d2ad65ad51f6 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:04.718917 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1159/2291] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:38:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:38:05.260285 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4ccec-cb71-41f6-8819-5f57dc115792 None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:05.260704 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4ccec-cb71-41f6-8819-5f57dc115792 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:05.261045 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4ccec-cb71-41f6-8819-5f57dc115792 None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:05.262515 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1133/2292] 10.222.0.49 () {54 vars in 961 bytes} [Mon Sep 26 10:38:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:38:05.271286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e370e-bfe6-4ca6-9ffc-6012453e1d3b None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:05.271569 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e370e-bfe6-4ca6-9ffc-6012453e1d3b None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:05.271847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e370e-bfe6-4ca6-9ffc-6012453e1d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:05.292462 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-389e370e-bfe6-4ca6-9ffc-6012453e1d3b None None] MFA Rules not processed for user `84b02e7351914cabb67462454b328d05`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:38:05.293665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-389e370e-bfe6-4ca6-9ffc-6012453e1d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:05.356864 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1160/2293] 10.222.0.49 () {58 vars in 1058 bytes} [Mon Sep 26 10:38:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:38:05.365072 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd4ccec-cb71-41f6-8819-5f57dc115792 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:05.378404 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd4ccec-cb71-41f6-8819-5f57dc115792 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:05.383298 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d7989-7f17-47f3-92f0-735a38ea60de service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84b02e7351914cabb67462454b328d05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:05.383884 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d7989-7f17-47f3-92f0-735a38ea60de service placement] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:05.384163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d7989-7f17-47f3-92f0-735a38ea60de service placement] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:05.384432 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d7989-7f17-47f3-92f0-735a38ea60de service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:05.386186 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-248d7989-7f17-47f3-92f0-735a38ea60de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:05.397328 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d7989-7f17-47f3-92f0-735a38ea60de service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:05.398950 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d7989-7f17-47f3-92f0-735a38ea60de service placement] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:05.408976 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1134/2294] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:38:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:38:06.375623 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e370e-bfe6-4ca6-9ffc-6012453e1d3b None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:06.394739 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e370e-bfe6-4ca6-9ffc-6012453e1d3b None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:06.401736 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262e437-a4f7-4043-8adf-172c7b2d7724 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:06.402652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262e437-a4f7-4043-8adf-172c7b2d7724 service nova] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:06.403097 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262e437-a4f7-4043-8adf-172c7b2d7724 service nova] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:06.403373 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262e437-a4f7-4043-8adf-172c7b2d7724 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:06.405324 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-c262e437-a4f7-4043-8adf-172c7b2d7724 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:06.427926 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262e437-a4f7-4043-8adf-172c7b2d7724 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:06.429706 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262e437-a4f7-4043-8adf-172c7b2d7724 service nova] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:06.444624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1161/2295] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:38:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:38:14.013832 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d7989-7f17-47f3-92f0-735a38ea60de service placement] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:14.026413 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d7989-7f17-47f3-92f0-735a38ea60de service placement] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:14.031676 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1c573-1ea7-458e-b6d5-0cd7ea1bb417 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:14.032252 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1c573-1ea7-458e-b6d5-0cd7ea1bb417 service nova] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:14.032526 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1c573-1ea7-458e-b6d5-0cd7ea1bb417 service nova] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:14.032792 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1c573-1ea7-458e-b6d5-0cd7ea1bb417 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:14.034487 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-0cc1c573-1ea7-458e-b6d5-0cd7ea1bb417 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:14.049452 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc1c573-1ea7-458e-b6d5-0cd7ea1bb417 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:14.051245 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc1c573-1ea7-458e-b6d5-0cd7ea1bb417 service nova] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:14.061927 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1135/2296] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:38:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:38:16.184692 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332764f-ef62-4db0-9cf6-12dcfd4de436 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:16.185152 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332764f-ef62-4db0-9cf6-12dcfd4de436 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:16.185359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332764f-ef62-4db0-9cf6-12dcfd4de436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:16.204982 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-2332764f-ef62-4db0-9cf6-12dcfd4de436 None None] MFA Rules not processed for user `9158557f1a2947e2815dc8d586d8b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:38:16.206286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-2332764f-ef62-4db0-9cf6-12dcfd4de436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:16.213891 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1162/2297] 10.222.0.49 () {58 vars in 967 bytes} [Mon Sep 26 10:38:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:38:16.263748 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1c573-1ea7-458e-b6d5-0cd7ea1bb417 service nova] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:16.275326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1c573-1ea7-458e-b6d5-0cd7ea1bb417 service nova] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:16.280241 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b89233-a998-49ea-95ab-ed464a7446d7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e6be34fc93a4551a83e5b8380188a6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:16.280843 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b89233-a998-49ea-95ab-ed464a7446d7 service swift] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:16.281110 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b89233-a998-49ea-95ab-ed464a7446d7 service swift] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:16.281376 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b89233-a998-49ea-95ab-ed464a7446d7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:16.290224 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b89233-a998-49ea-95ab-ed464a7446d7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:16.291778 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b89233-a998-49ea-95ab-ed464a7446d7 service swift] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:16.300126 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1136/2298] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 10:38:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 10:38:16.510269 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332764f-ef62-4db0-9cf6-12dcfd4de436 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:16.511741 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-2332764f-ef62-4db0-9cf6-12dcfd4de436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:16.526419 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332764f-ef62-4db0-9cf6-12dcfd4de436 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:16.530362 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ec943-bcef-420d-be3f-f91efe8260e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:16.531129 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ec943-bcef-420d-be3f-f91efe8260e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:16.531672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ec943-bcef-420d-be3f-f91efe8260e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:16.532280 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ec943-bcef-420d-be3f-f91efe8260e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:16.534289 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-fe4ec943-bcef-420d-be3f-f91efe8260e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:16.601844 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ec943-bcef-420d-be3f-f91efe8260e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:16.603600 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ec943-bcef-420d-be3f-f91efe8260e4 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:16.614232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1163/2299] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:38:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:38:20.207260 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b89233-a998-49ea-95ab-ed464a7446d7 service swift] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:20.210324 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-04b89233-a998-49ea-95ab-ed464a7446d7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:20.235324 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b89233-a998-49ea-95ab-ed464a7446d7 service swift] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:20.243127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b4964d-ecdc-480b-b46f-7987ac5b7265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:20.243766 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4964d-ecdc-480b-b46f-7987ac5b7265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:20.244046 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4964d-ecdc-480b-b46f-7987ac5b7265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:20.244316 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4964d-ecdc-480b-b46f-7987ac5b7265 admin admin] PATH_INFO: `/v3/users/5085a918c6b74488b3dd70c1fb257381` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:20.245595 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b4964d-ecdc-480b-b46f-7987ac5b7265 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5085a918c6b74488b3dd70c1fb257381)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:20.247610 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b4964d-ecdc-480b-b46f-7987ac5b7265 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:20.325133 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-05b4964d-ecdc-480b-b46f-7987ac5b7265 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5085a918c6b74488b3dd70c1fb257381'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:20.326039 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-05b4964d-ecdc-480b-b46f-7987ac5b7265 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5085a918c6b74488b3dd70c1fb257381'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:20.326803 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-05b4964d-ecdc-480b-b46f-7987ac5b7265 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5085a918c6b74488b3dd70c1fb257381'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:20.356660 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-05b4964d-ecdc-480b-b46f-7987ac5b7265 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5085a918c6b74488b3dd70c1fb257381'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:20.382104 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-05b4964d-ecdc-480b-b46f-7987ac5b7265 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5085a918c6b74488b3dd70c1fb257381'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:20.408106 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1137/2300] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:38:20 2022] DELETE /identity/v3/users/5085a918c6b74488b3dd70c1fb257381 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:38:20.762753 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ec943-bcef-420d-be3f-f91efe8260e4 service neutron] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:20.774264 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ec943-bcef-420d-be3f-f91efe8260e4 service neutron] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:20.844489 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:20.872436 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438083f8-10d7-44d9-a6a1-bb1172e296c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:20.873406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438083f8-10d7-44d9-a6a1-bb1172e296c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:20.873792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438083f8-10d7-44d9-a6a1-bb1172e296c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:20.874194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438083f8-10d7-44d9-a6a1-bb1172e296c1 admin admin] PATH_INFO: `/v3/projects/615e5825389c4f1d96e6398bf1dab735` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:20.875812 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438083f8-10d7-44d9-a6a1-bb1172e296c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=615e5825389c4f1d96e6398bf1dab735)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:20.880590 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438083f8-10d7-44d9-a6a1-bb1172e296c1 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:20.953079 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-438083f8-10d7-44d9-a6a1-bb1172e296c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '615e5825389c4f1d96e6398bf1dab735', 'request_id': 'req-438083f8-10d7-44d9-a6a1-bb1172e296c1'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:20.959599 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-438083f8-10d7-44d9-a6a1-bb1172e296c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '615e5825389c4f1d96e6398bf1dab735', 'request_id': 'req-438083f8-10d7-44d9-a6a1-bb1172e296c1'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:20.968584 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-438083f8-10d7-44d9-a6a1-bb1172e296c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '615e5825389c4f1d96e6398bf1dab735', 'request_id': 'req-438083f8-10d7-44d9-a6a1-bb1172e296c1'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:20.984851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-438083f8-10d7-44d9-a6a1-bb1172e296c1 admin admin] The token cache is being invalidate because project 615e5825389c4f1d96e6398bf1dab735 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:38:20.985177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-438083f8-10d7-44d9-a6a1-bb1172e296c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:20.986037 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-438083f8-10d7-44d9-a6a1-bb1172e296c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:20.988378 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1164/2301] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:38:20 2022] DELETE /identity/v3/projects/615e5825389c4f1d96e6398bf1dab735 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:38:20.994724 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe80978-e752-4b49-bbf3-9d31aeca5f6c None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:20.994982 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe80978-e752-4b49-bbf3-9d31aeca5f6c None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:20.995241 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe80978-e752-4b49-bbf3-9d31aeca5f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:21.015765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-1fe80978-e752-4b49-bbf3-9d31aeca5f6c None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:38:21.017015 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-1fe80978-e752-4b49-bbf3-9d31aeca5f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:21.079256 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1138/2302] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:38:21.086921 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c234378-5c8d-4eb3-a982-aec2de44d47a None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:21.087202 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c234378-5c8d-4eb3-a982-aec2de44d47a None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:21.087655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c234378-5c8d-4eb3-a982-aec2de44d47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:21.108029 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-7c234378-5c8d-4eb3-a982-aec2de44d47a None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:38:21.109246 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-7c234378-5c8d-4eb3-a982-aec2de44d47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:21.117850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1165/2303] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:38:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:38:21.122297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe80978-e752-4b49-bbf3-9d31aeca5f6c None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:21.132299 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe80978-e752-4b49-bbf3-9d31aeca5f6c None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:21.136803 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aef77d-737c-427b-acab-b294fe4f488f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:21.137392 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aef77d-737c-427b-acab-b294fe4f488f admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:21.137669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aef77d-737c-427b-acab-b294fe4f488f admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:21.137940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aef77d-737c-427b-acab-b294fe4f488f admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:21.138620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aef77d-737c-427b-acab-b294fe4f488f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:38:21.138921 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aef77d-737c-427b-acab-b294fe4f488f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:21.140521 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aef77d-737c-427b-acab-b294fe4f488f admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:21.156165 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1139/2304] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:38:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:38:21.157362 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c234378-5c8d-4eb3-a982-aec2de44d47a None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:21.167894 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c234378-5c8d-4eb3-a982-aec2de44d47a None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:21.172618 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0bd38a-9b5b-4aa1-afc3-215ae863e32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:21.173227 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0bd38a-9b5b-4aa1-afc3-215ae863e32b admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:21.173514 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0bd38a-9b5b-4aa1-afc3-215ae863e32b admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:21.173792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0bd38a-9b5b-4aa1-afc3-215ae863e32b admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:21.174462 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0bd38a-9b5b-4aa1-afc3-215ae863e32b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:21.176441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0bd38a-9b5b-4aa1-afc3-215ae863e32b admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:21.205127 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1166/2305] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:38:21 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:38:21.209229 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aef77d-737c-427b-acab-b294fe4f488f admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:21.219475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aef77d-737c-427b-acab-b294fe4f488f admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:21.229111 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:21.281461 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca2694-df41-4982-a490-e382814b50f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:21.282372 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca2694-df41-4982-a490-e382814b50f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:21.282652 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca2694-df41-4982-a490-e382814b50f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:21.282925 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca2694-df41-4982-a490-e382814b50f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:21.283562 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ca2694-df41-4982-a490-e382814b50f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:21.285477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ca2694-df41-4982-a490-e382814b50f1 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:21.314802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1140/2306] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:38:21 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:38:21.316075 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0bd38a-9b5b-4aa1-afc3-215ae863e32b admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:21.327201 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0bd38a-9b5b-4aa1-afc3-215ae863e32b admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:21.332067 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5bac43-1688-4f08-884a-ee475229d90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:21.333116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5bac43-1688-4f08-884a-ee475229d90a admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:21.333404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5bac43-1688-4f08-884a-ee475229d90a admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:21.333783 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5bac43-1688-4f08-884a-ee475229d90a admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:21.334495 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5bac43-1688-4f08-884a-ee475229d90a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:21.336262 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5bac43-1688-4f08-884a-ee475229d90a admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:21.346903 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1167/2307] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:38:21.351593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca2694-df41-4982-a490-e382814b50f1 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:21.362589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca2694-df41-4982-a490-e382814b50f1 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:21.369211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092535c-7f5a-482b-829b-f031fa5b0935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:21.369838 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092535c-7f5a-482b-829b-f031fa5b0935 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:21.370154 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092535c-7f5a-482b-829b-f031fa5b0935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:21.370479 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092535c-7f5a-482b-829b-f031fa5b0935 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:21.371150 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e092535c-7f5a-482b-829b-f031fa5b0935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:21.372755 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e092535c-7f5a-482b-829b-f031fa5b0935 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:21.389673 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1141/2308] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:38:21.393951 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5bac43-1688-4f08-884a-ee475229d90a admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:21.405892 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5bac43-1688-4f08-884a-ee475229d90a admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:21.411706 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e53ef-38f2-4097-b048-2526481387fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:21.411913 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e53ef-38f2-4097-b048-2526481387fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:21.412309 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e53ef-38f2-4097-b048-2526481387fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:21.412654 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e53ef-38f2-4097-b048-2526481387fb admin admin] PATH_INFO: `/v3/projects/1219307a321f438c8efa9f1268974e6e/users/3d79db1ac5f14d908ea33c02f12a721d/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:21.430812 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9e53ef-38f2-4097-b048-2526481387fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1219307a321f438c8efa9f1268974e6e, user_id=3d79db1ac5f14d908ea33c02f12a721d, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:21.433537 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9e53ef-38f2-4097-b048-2526481387fb admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:21.448633 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1168/2309] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:38:21 2022] PUT /identity/v3/projects/1219307a321f438c8efa9f1268974e6e/users/3d79db1ac5f14d908ea33c02f12a721d/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:38:21.460951 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503814e1-32ad-48c3-b1e3-09eb0b6e6359 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:21.461232 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503814e1-32ad-48c3-b1e3-09eb0b6e6359 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:21.461527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503814e1-32ad-48c3-b1e3-09eb0b6e6359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:21.493621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-503814e1-32ad-48c3-b1e3-09eb0b6e6359 None None] MFA Rules not processed for user `3d79db1ac5f14d908ea33c02f12a721d`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:38:21.495131 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-503814e1-32ad-48c3-b1e3-09eb0b6e6359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:21.589274 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1142/2310] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:38:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:38:21.599153 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e53ef-38f2-4097-b048-2526481387fb admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:21.601375 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-ff9e53ef-38f2-4097-b048-2526481387fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:21.688167 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e53ef-38f2-4097-b048-2526481387fb admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:21.693060 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9bc675-8c19-4f04-ba0c-b01dcf8665b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:21.693716 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bc675-8c19-4f04-ba0c-b01dcf8665b6 service nova] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:21.693978 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bc675-8c19-4f04-ba0c-b01dcf8665b6 service nova] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:21.694279 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bc675-8c19-4f04-ba0c-b01dcf8665b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:21.707486 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:21.712538 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9bc675-8c19-4f04-ba0c-b01dcf8665b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:21.715079 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9bc675-8c19-4f04-ba0c-b01dcf8665b6 service nova] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:21.730241 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1169/2311] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:38:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:38:24.579625 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503814e1-32ad-48c3-b1e3-09eb0b6e6359 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:24.581242 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-503814e1-32ad-48c3-b1e3-09eb0b6e6359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:24.624113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9bc675-8c19-4f04-ba0c-b01dcf8665b6 service nova] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:24.653900 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503814e1-32ad-48c3-b1e3-09eb0b6e6359 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:24.659182 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c43b9-f6ab-460d-a598-1835f37957a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:24.659815 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c43b9-f6ab-460d-a598-1835f37957a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:24.660091 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c43b9-f6ab-460d-a598-1835f37957a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:24.660360 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c43b9-f6ab-460d-a598-1835f37957a9 admin admin] PATH_INFO: `/v3/users/681edd9335a644a69a497a5c528920a8` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:24.662010 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57c43b9-f6ab-460d-a598-1835f37957a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=681edd9335a644a69a497a5c528920a8)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:24.664130 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57c43b9-f6ab-460d-a598-1835f37957a9 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:24.686836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:24.688105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9bc675-8c19-4f04-ba0c-b01dcf8665b6 service nova] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:24.693685 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352cabe8-a666-4135-aa5b-0f43312ef37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:24.694345 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352cabe8-a666-4135-aa5b-0f43312ef37d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:24.694616 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352cabe8-a666-4135-aa5b-0f43312ef37d admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:24.694883 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352cabe8-a666-4135-aa5b-0f43312ef37d admin admin] PATH_INFO: `/v3/users/3d79db1ac5f14d908ea33c02f12a721d` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:24.696196 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352cabe8-a666-4135-aa5b-0f43312ef37d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d79db1ac5f14d908ea33c02f12a721d)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:24.698416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352cabe8-a666-4135-aa5b-0f43312ef37d admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:24.716638 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e57c43b9-f6ab-460d-a598-1835f37957a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '681edd9335a644a69a497a5c528920a8'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:24.717316 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e57c43b9-f6ab-460d-a598-1835f37957a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '681edd9335a644a69a497a5c528920a8'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:24.717922 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e57c43b9-f6ab-460d-a598-1835f37957a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '681edd9335a644a69a497a5c528920a8'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:24.726674 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e57c43b9-f6ab-460d-a598-1835f37957a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '681edd9335a644a69a497a5c528920a8'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:24.740381 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e57c43b9-f6ab-460d-a598-1835f37957a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '681edd9335a644a69a497a5c528920a8'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:24.747549 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-352cabe8-a666-4135-aa5b-0f43312ef37d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d79db1ac5f14d908ea33c02f12a721d'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:24.748283 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-352cabe8-a666-4135-aa5b-0f43312ef37d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d79db1ac5f14d908ea33c02f12a721d'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:24.753124 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1143/2312] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:38:24 2022] DELETE /identity/v3/users/681edd9335a644a69a497a5c528920a8 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:38:24.756301 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-352cabe8-a666-4135-aa5b-0f43312ef37d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d79db1ac5f14d908ea33c02f12a721d'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:24.768776 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-352cabe8-a666-4135-aa5b-0f43312ef37d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d79db1ac5f14d908ea33c02f12a721d'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:24.769478 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-352cabe8-a666-4135-aa5b-0f43312ef37d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d79db1ac5f14d908ea33c02f12a721d'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:24.783909 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1170/2313] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:38:24 2022] DELETE /identity/v3/users/3d79db1ac5f14d908ea33c02f12a721d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:38:24.788316 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd907d72-f9c7-4dd0-9089-5a755ba69c4a None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:24.788776 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd907d72-f9c7-4dd0-9089-5a755ba69c4a None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:24.789056 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd907d72-f9c7-4dd0-9089-5a755ba69c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:24.809668 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-bd907d72-f9c7-4dd0-9089-5a755ba69c4a None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:38:24.810919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-bd907d72-f9c7-4dd0-9089-5a755ba69c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:24.872493 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1144/2314] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:38:24.882796 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352cabe8-a666-4135-aa5b-0f43312ef37d admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:24.884315 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-352cabe8-a666-4135-aa5b-0f43312ef37d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:24.939502 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd907d72-f9c7-4dd0-9089-5a755ba69c4a None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:24.941106 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-bd907d72-f9c7-4dd0-9089-5a755ba69c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:24.945601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352cabe8-a666-4135-aa5b-0f43312ef37d admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:24.949456 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e78bfee-365c-4181-9cd1-c7baad9cab1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:24.950423 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78bfee-365c-4181-9cd1-c7baad9cab1e service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:24.950766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78bfee-365c-4181-9cd1-c7baad9cab1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:24.951104 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78bfee-365c-4181-9cd1-c7baad9cab1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:24.955112 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd907d72-f9c7-4dd0-9089-5a755ba69c4a None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:24.961356 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e78bfee-365c-4181-9cd1-c7baad9cab1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:24.962547 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733098c2-adf8-422a-adc2-e4e3fbe16d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:24.962820 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e78bfee-365c-4181-9cd1-c7baad9cab1e service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:24.963449 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733098c2-adf8-422a-adc2-e4e3fbe16d2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:24.963724 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733098c2-adf8-422a-adc2-e4e3fbe16d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:24.963991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733098c2-adf8-422a-adc2-e4e3fbe16d2c admin admin] PATH_INFO: `/v3/projects/6bb20c7af7544a06adaa9b5e47b03cd1` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:24.965121 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733098c2-adf8-422a-adc2-e4e3fbe16d2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bb20c7af7544a06adaa9b5e47b03cd1)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:24.967431 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733098c2-adf8-422a-adc2-e4e3fbe16d2c admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:24.974827 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1171/2315] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:38:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:38:25.024490 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-733098c2-adf8-422a-adc2-e4e3fbe16d2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bb20c7af7544a06adaa9b5e47b03cd1', 'request_id': 'req-733098c2-adf8-422a-adc2-e4e3fbe16d2c'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:25.031292 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-733098c2-adf8-422a-adc2-e4e3fbe16d2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bb20c7af7544a06adaa9b5e47b03cd1', 'request_id': 'req-733098c2-adf8-422a-adc2-e4e3fbe16d2c'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:25.037353 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-733098c2-adf8-422a-adc2-e4e3fbe16d2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bb20c7af7544a06adaa9b5e47b03cd1', 'request_id': 'req-733098c2-adf8-422a-adc2-e4e3fbe16d2c'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:25.049614 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-733098c2-adf8-422a-adc2-e4e3fbe16d2c admin admin] The token cache is being invalidate because project 6bb20c7af7544a06adaa9b5e47b03cd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:38:25.049903 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-733098c2-adf8-422a-adc2-e4e3fbe16d2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:25.050572 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-733098c2-adf8-422a-adc2-e4e3fbe16d2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:25.052326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1145/2316] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:38:24 2022] DELETE /identity/v3/projects/6bb20c7af7544a06adaa9b5e47b03cd1 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:38:25.059998 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e78bfee-365c-4181-9cd1-c7baad9cab1e service neutron] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:25.071499 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e78bfee-365c-4181-9cd1-c7baad9cab1e service neutron] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:25.085281 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:25.136176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b06c97-628f-44de-9a79-95b8cafb02d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:25.136812 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b06c97-628f-44de-9a79-95b8cafb02d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:25.137093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b06c97-628f-44de-9a79-95b8cafb02d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:25.137659 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b06c97-628f-44de-9a79-95b8cafb02d5 admin admin] PATH_INFO: `/v3/users/80c4c3b2a04f44b49021fb61b4c53826` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:25.138523 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b06c97-628f-44de-9a79-95b8cafb02d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80c4c3b2a04f44b49021fb61b4c53826)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:25.140458 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b06c97-628f-44de-9a79-95b8cafb02d5 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:25.190771 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-36b06c97-628f-44de-9a79-95b8cafb02d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80c4c3b2a04f44b49021fb61b4c53826'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:25.191802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-36b06c97-628f-44de-9a79-95b8cafb02d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80c4c3b2a04f44b49021fb61b4c53826'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:25.199568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-36b06c97-628f-44de-9a79-95b8cafb02d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80c4c3b2a04f44b49021fb61b4c53826'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:25.211427 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-36b06c97-628f-44de-9a79-95b8cafb02d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80c4c3b2a04f44b49021fb61b4c53826'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:25.212163 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-36b06c97-628f-44de-9a79-95b8cafb02d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80c4c3b2a04f44b49021fb61b4c53826'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:25.226160 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1172/2317] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:38:25 2022] DELETE /identity/v3/users/80c4c3b2a04f44b49021fb61b4c53826 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:38:25.277504 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733098c2-adf8-422a-adc2-e4e3fbe16d2c admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:25.279131 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-733098c2-adf8-422a-adc2-e4e3fbe16d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:25.298452 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733098c2-adf8-422a-adc2-e4e3fbe16d2c admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:25.329087 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:25.372014 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c825084-dedd-4666-a7e1-247559852a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:25.372653 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c825084-dedd-4666-a7e1-247559852a86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:25.372931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c825084-dedd-4666-a7e1-247559852a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:25.373203 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c825084-dedd-4666-a7e1-247559852a86 admin admin] PATH_INFO: `/v3/projects/1219307a321f438c8efa9f1268974e6e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:25.374350 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c825084-dedd-4666-a7e1-247559852a86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1219307a321f438c8efa9f1268974e6e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:25.376347 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c825084-dedd-4666-a7e1-247559852a86 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:25.413389 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b06c97-628f-44de-9a79-95b8cafb02d5 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:25.415327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-36b06c97-628f-44de-9a79-95b8cafb02d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:25.429447 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b06c97-628f-44de-9a79-95b8cafb02d5 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:25.432391 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-8c825084-dedd-4666-a7e1-247559852a86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1219307a321f438c8efa9f1268974e6e', 'request_id': 'req-8c825084-dedd-4666-a7e1-247559852a86'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:25.441178 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-8c825084-dedd-4666-a7e1-247559852a86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1219307a321f438c8efa9f1268974e6e', 'request_id': 'req-8c825084-dedd-4666-a7e1-247559852a86'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:25.448444 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-8c825084-dedd-4666-a7e1-247559852a86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1219307a321f438c8efa9f1268974e6e', 'request_id': 'req-8c825084-dedd-4666-a7e1-247559852a86'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:25.464964 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-8c825084-dedd-4666-a7e1-247559852a86 admin admin] The token cache is being invalidate because project 1219307a321f438c8efa9f1268974e6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:38:25.465257 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-8c825084-dedd-4666-a7e1-247559852a86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:25.465887 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-8c825084-dedd-4666-a7e1-247559852a86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:25.467658 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1146/2318] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:38:25 2022] DELETE /identity/v3/projects/1219307a321f438c8efa9f1268974e6e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:38:25.472015 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a321c7f-4f65-4719-a58f-fd336e2a7228 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:25.472291 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a321c7f-4f65-4719-a58f-fd336e2a7228 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:25.472565 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a321c7f-4f65-4719-a58f-fd336e2a7228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:25.494149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-9a321c7f-4f65-4719-a58f-fd336e2a7228 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:38:25.494329 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392a33c-7f92-4cd0-9fcf-cf027d93e7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:25.494949 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392a33c-7f92-4cd0-9fcf-cf027d93e7e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:25.495350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392a33c-7f92-4cd0-9fcf-cf027d93e7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:25.495449 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-9a321c7f-4f65-4719-a58f-fd336e2a7228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:25.495799 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392a33c-7f92-4cd0-9fcf-cf027d93e7e9 admin admin] PATH_INFO: `/v3/projects/bbb6530b068f4adba1992dd64f303e79` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:25.497016 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392a33c-7f92-4cd0-9fcf-cf027d93e7e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbb6530b068f4adba1992dd64f303e79)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:25.499027 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392a33c-7f92-4cd0-9fcf-cf027d93e7e9 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:25.503484 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1147/2319] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:38:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:38:25.511156 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901eb772-794d-4aaa-a142-2cf537dfbded None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:25.511439 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901eb772-794d-4aaa-a142-2cf537dfbded None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:25.511707 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901eb772-794d-4aaa-a142-2cf537dfbded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:25.531164 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-901eb772-794d-4aaa-a142-2cf537dfbded None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:38:25.532376 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-901eb772-794d-4aaa-a142-2cf537dfbded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:25.541860 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:25.551040 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-5392a33c-7f92-4cd0-9fcf-cf027d93e7e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbb6530b068f4adba1992dd64f303e79', 'request_id': 'req-5392a33c-7f92-4cd0-9fcf-cf027d93e7e9'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:25.557903 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-5392a33c-7f92-4cd0-9fcf-cf027d93e7e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbb6530b068f4adba1992dd64f303e79', 'request_id': 'req-5392a33c-7f92-4cd0-9fcf-cf027d93e7e9'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:25.566640 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-5392a33c-7f92-4cd0-9fcf-cf027d93e7e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbb6530b068f4adba1992dd64f303e79', 'request_id': 'req-5392a33c-7f92-4cd0-9fcf-cf027d93e7e9'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:25.577666 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-5392a33c-7f92-4cd0-9fcf-cf027d93e7e9 admin admin] The token cache is being invalidate because project bbb6530b068f4adba1992dd64f303e79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:38:25.578041 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-5392a33c-7f92-4cd0-9fcf-cf027d93e7e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:25.578730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-5392a33c-7f92-4cd0-9fcf-cf027d93e7e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:25.580704 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1173/2320] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:38:25 2022] DELETE /identity/v3/projects/bbb6530b068f4adba1992dd64f303e79 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:38:25.597511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1148/2321] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:38:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:38:25.602641 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392a33c-7f92-4cd0-9fcf-cf027d93e7e9 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:25.614713 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392a33c-7f92-4cd0-9fcf-cf027d93e7e9 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:25.619816 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32f846-73d9-49a1-bf06-f0d049f7e3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:25.620445 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32f846-73d9-49a1-bf06-f0d049f7e3f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:25.620723 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32f846-73d9-49a1-bf06-f0d049f7e3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:25.620992 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32f846-73d9-49a1-bf06-f0d049f7e3f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:25.621686 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b32f846-73d9-49a1-bf06-f0d049f7e3f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:38:25.622089 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b32f846-73d9-49a1-bf06-f0d049f7e3f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:25.623760 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b32f846-73d9-49a1-bf06-f0d049f7e3f3 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:25.640968 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1174/2322] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:38:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:38:25.642136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901eb772-794d-4aaa-a142-2cf537dfbded None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:25.652117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901eb772-794d-4aaa-a142-2cf537dfbded None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:25.657045 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d3309-d060-4584-91ae-863bbd6486b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:25.657671 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d3309-d060-4584-91ae-863bbd6486b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:25.657950 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d3309-d060-4584-91ae-863bbd6486b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:25.658256 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d3309-d060-4584-91ae-863bbd6486b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:25.658918 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d3309-d060-4584-91ae-863bbd6486b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:25.661431 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d3309-d060-4584-91ae-863bbd6486b4 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:25.696322 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1149/2323] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:38:25 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:38:25.703477 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32f846-73d9-49a1-bf06-f0d049f7e3f3 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:25.714422 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32f846-73d9-49a1-bf06-f0d049f7e3f3 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:25.779520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056c085-92ef-4a6a-930a-baa23bf5a636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:25.780278 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056c085-92ef-4a6a-930a-baa23bf5a636 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:25.780657 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056c085-92ef-4a6a-930a-baa23bf5a636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:25.781025 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056c085-92ef-4a6a-930a-baa23bf5a636 admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:25.781823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8056c085-92ef-4a6a-930a-baa23bf5a636 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:25.783954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8056c085-92ef-4a6a-930a-baa23bf5a636 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:25.821766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1175/2324] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:38:25 2022] POST /identity/v3/users => generated 294 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:38:25.823169 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d3309-d060-4584-91ae-863bbd6486b4 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:25.835154 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d3309-d060-4584-91ae-863bbd6486b4 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:25.840007 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026ead5-e07f-4e5c-93b5-5d89c11700d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:25.840633 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026ead5-e07f-4e5c-93b5-5d89c11700d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:25.840911 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026ead5-e07f-4e5c-93b5-5d89c11700d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:25.841181 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026ead5-e07f-4e5c-93b5-5d89c11700d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:25.841830 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f026ead5-e07f-4e5c-93b5-5d89c11700d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:25.843475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f026ead5-e07f-4e5c-93b5-5d89c11700d0 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:25.855206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1150/2325] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:38:25.859211 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056c085-92ef-4a6a-930a-baa23bf5a636 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:25.869225 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056c085-92ef-4a6a-930a-baa23bf5a636 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:25.873951 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc3491d-a007-49bb-b813-7ab36297512f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:25.874568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3491d-a007-49bb-b813-7ab36297512f admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:25.874842 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3491d-a007-49bb-b813-7ab36297512f admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:25.875112 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3491d-a007-49bb-b813-7ab36297512f admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:25.875743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc3491d-a007-49bb-b813-7ab36297512f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:25.877327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc3491d-a007-49bb-b813-7ab36297512f admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:25.889779 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1176/2326] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:38:25.893968 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026ead5-e07f-4e5c-93b5-5d89c11700d0 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:25.904609 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026ead5-e07f-4e5c-93b5-5d89c11700d0 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:25.909373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62c9fd-a71a-49d0-aaab-21f163801e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:25.910012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62c9fd-a71a-49d0-aaab-21f163801e7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:25.910315 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62c9fd-a71a-49d0-aaab-21f163801e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:25.910584 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62c9fd-a71a-49d0-aaab-21f163801e7a admin admin] PATH_INFO: `/v3/projects/589d9929a2144458951bc1ea280e26e0/users/2303d18914d54825bdd33c7a06a739da/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:25.923162 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:25.926569 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d62c9fd-a71a-49d0-aaab-21f163801e7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=589d9929a2144458951bc1ea280e26e0, user_id=2303d18914d54825bdd33c7a06a739da, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:25.929579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d62c9fd-a71a-49d0-aaab-21f163801e7a admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:25.943394 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1151/2327] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:38:25 2022] PUT /identity/v3/projects/589d9929a2144458951bc1ea280e26e0/users/2303d18914d54825bdd33c7a06a739da/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:38:25.953584 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aef974-642e-40d9-83a1-ae4e85d8eade None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:25.953865 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aef974-642e-40d9-83a1-ae4e85d8eade None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:25.954169 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aef974-642e-40d9-83a1-ae4e85d8eade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:25.982427 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-87aef974-642e-40d9-83a1-ae4e85d8eade None None] MFA Rules not processed for user `2303d18914d54825bdd33c7a06a739da`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:38:25.983626 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-87aef974-642e-40d9-83a1-ae4e85d8eade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:26.047004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1177/2328] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:38:25 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:38:26.051418 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62c9fd-a71a-49d0-aaab-21f163801e7a admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:26.113594 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62c9fd-a71a-49d0-aaab-21f163801e7a admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:26.118405 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0d219-5cde-483b-b303-be0c1a9cb4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:26.119026 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0d219-5cde-483b-b303-be0c1a9cb4a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:26.119308 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0d219-5cde-483b-b303-be0c1a9cb4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:26.119578 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0d219-5cde-483b-b303-be0c1a9cb4a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:26.120203 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a0d219-5cde-483b-b303-be0c1a9cb4a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:26.122129 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a0d219-5cde-483b-b303-be0c1a9cb4a0 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:26.148717 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1152/2329] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:38:26 2022] POST /identity/v3/projects => generated 389 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:38:26.152766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aef974-642e-40d9-83a1-ae4e85d8eade None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:26.164035 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aef974-642e-40d9-83a1-ae4e85d8eade None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:26.181714 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:26.342759 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ccc3a1-ab13-4daf-b536-2051e9b9c793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:26.343275 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ccc3a1-ab13-4daf-b536-2051e9b9c793 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:26.343550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ccc3a1-ab13-4daf-b536-2051e9b9c793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:26.343820 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ccc3a1-ab13-4daf-b536-2051e9b9c793 admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:26.344452 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ccc3a1-ab13-4daf-b536-2051e9b9c793 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:26.346381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ccc3a1-ab13-4daf-b536-2051e9b9c793 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:26.367661 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1178/2330] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:38:26 2022] POST /identity/v3/users => generated 301 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:38:26.371468 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0d219-5cde-483b-b303-be0c1a9cb4a0 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:26.382341 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0d219-5cde-483b-b303-be0c1a9cb4a0 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:26.386852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b62d17-8b7d-4b92-8c1b-c0ea1cfa4499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:26.387436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b62d17-8b7d-4b92-8c1b-c0ea1cfa4499 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:26.387709 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b62d17-8b7d-4b92-8c1b-c0ea1cfa4499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:26.387976 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b62d17-8b7d-4b92-8c1b-c0ea1cfa4499 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:26.388639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b62d17-8b7d-4b92-8c1b-c0ea1cfa4499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:26.390253 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b62d17-8b7d-4b92-8c1b-c0ea1cfa4499 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:26.401057 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1153/2331] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:38:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:38:26.405052 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ccc3a1-ab13-4daf-b536-2051e9b9c793 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:26.415173 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ccc3a1-ab13-4daf-b536-2051e9b9c793 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:26.420296 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83543fc-8481-427f-af8e-1f1a1ffe78c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:26.420940 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83543fc-8481-427f-af8e-1f1a1ffe78c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:26.421264 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83543fc-8481-427f-af8e-1f1a1ffe78c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:26.421554 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83543fc-8481-427f-af8e-1f1a1ffe78c4 admin admin] PATH_INFO: `/v3/projects/64575736030f4f219dd8444e7bf3378b/users/f3c9951be9dc4a41a76280845b345e6f/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:26.436622 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83543fc-8481-427f-af8e-1f1a1ffe78c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64575736030f4f219dd8444e7bf3378b, user_id=f3c9951be9dc4a41a76280845b345e6f, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:26.439392 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83543fc-8481-427f-af8e-1f1a1ffe78c4 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:26.460174 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1179/2332] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:38:26 2022] PUT /identity/v3/projects/64575736030f4f219dd8444e7bf3378b/users/f3c9951be9dc4a41a76280845b345e6f/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:38:26.464042 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b62d17-8b7d-4b92-8c1b-c0ea1cfa4499 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:26.474603 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b62d17-8b7d-4b92-8c1b-c0ea1cfa4499 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:26.542591 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294e31e-2663-43d0-a6cf-b6d0c516e41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:26.543206 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294e31e-2663-43d0-a6cf-b6d0c516e41e admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:26.543482 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294e31e-2663-43d0-a6cf-b6d0c516e41e admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:26.543751 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294e31e-2663-43d0-a6cf-b6d0c516e41e admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:26.544393 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b294e31e-2663-43d0-a6cf-b6d0c516e41e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:26.545987 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b294e31e-2663-43d0-a6cf-b6d0c516e41e admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:26.557503 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1154/2333] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:38:26 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:38:26.561784 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83543fc-8481-427f-af8e-1f1a1ffe78c4 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:26.572713 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83543fc-8481-427f-af8e-1f1a1ffe78c4 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:26.577758 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e17763-1a96-443b-afa3-bacb8468e93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:26.578481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e17763-1a96-443b-afa3-bacb8468e93d admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:26.579128 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e17763-1a96-443b-afa3-bacb8468e93d admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:26.579128 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e17763-1a96-443b-afa3-bacb8468e93d admin admin] PATH_INFO: `/v3/projects/64575736030f4f219dd8444e7bf3378b/users/f3c9951be9dc4a41a76280845b345e6f/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:26.581072 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e17763-1a96-443b-afa3-bacb8468e93d admin admin] RBAC: Authorizing `identity:create_grant(project_id=64575736030f4f219dd8444e7bf3378b, user_id=f3c9951be9dc4a41a76280845b345e6f, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:26.584618 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e17763-1a96-443b-afa3-bacb8468e93d admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:26.595919 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1180/2334] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:38:26 2022] PUT /identity/v3/projects/64575736030f4f219dd8444e7bf3378b/users/f3c9951be9dc4a41a76280845b345e6f/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:38:26.607889 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfaeea6-c6db-4695-8c12-d1f0e7995d5f None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:26.608415 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfaeea6-c6db-4695-8c12-d1f0e7995d5f None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:26.608415 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfaeea6-c6db-4695-8c12-d1f0e7995d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:26.638836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-1bfaeea6-c6db-4695-8c12-d1f0e7995d5f None None] MFA Rules not processed for user `f3c9951be9dc4a41a76280845b345e6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:38:26.640203 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-1bfaeea6-c6db-4695-8c12-d1f0e7995d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:26.700614 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:26.730393 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1155/2335] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:38:26 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:38:26.738008 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e17763-1a96-443b-afa3-bacb8468e93d admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:26.740337 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-07e17763-1a96-443b-afa3-bacb8468e93d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:26.800174 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e17763-1a96-443b-afa3-bacb8468e93d admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:26.803848 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6bceb-bac0-45b9-9edb-c4951c8bdd28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:26.804494 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6bceb-bac0-45b9-9edb-c4951c8bdd28 service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:26.804750 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6bceb-bac0-45b9-9edb-c4951c8bdd28 service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:26.805014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6bceb-bac0-45b9-9edb-c4951c8bdd28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:26.814234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b6bceb-bac0-45b9-9edb-c4951c8bdd28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:26.815643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b6bceb-bac0-45b9-9edb-c4951c8bdd28 service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:26.824964 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1181/2336] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:38:26 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:38:26.961220 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfaeea6-c6db-4695-8c12-d1f0e7995d5f None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:27.032402 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfaeea6-c6db-4695-8c12-d1f0e7995d5f None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:27.037700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25208c6-1475-4680-8e55-922799d16e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:27.038400 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25208c6-1475-4680-8e55-922799d16e25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:27.038677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25208c6-1475-4680-8e55-922799d16e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:27.038976 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25208c6-1475-4680-8e55-922799d16e25 admin admin] PATH_INFO: `/v3/users/2303d18914d54825bdd33c7a06a739da` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:27.040129 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25208c6-1475-4680-8e55-922799d16e25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2303d18914d54825bdd33c7a06a739da)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:27.042130 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25208c6-1475-4680-8e55-922799d16e25 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:27.089462 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-a25208c6-1475-4680-8e55-922799d16e25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2303d18914d54825bdd33c7a06a739da'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:27.090150 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-a25208c6-1475-4680-8e55-922799d16e25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2303d18914d54825bdd33c7a06a739da'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:27.090753 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-a25208c6-1475-4680-8e55-922799d16e25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2303d18914d54825bdd33c7a06a739da'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:27.097455 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-a25208c6-1475-4680-8e55-922799d16e25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2303d18914d54825bdd33c7a06a739da'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:27.109289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-a25208c6-1475-4680-8e55-922799d16e25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2303d18914d54825bdd33c7a06a739da'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:27.121816 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1156/2337] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:38:26 2022] DELETE /identity/v3/users/2303d18914d54825bdd33c7a06a739da => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:38:27.126354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408930c-159e-4a86-b950-530f481d5df6 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:27.126637 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408930c-159e-4a86-b950-530f481d5df6 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:27.126906 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408930c-159e-4a86-b950-530f481d5df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:27.147357 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-6408930c-159e-4a86-b950-530f481d5df6 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:38:27.148541 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-6408930c-159e-4a86-b950-530f481d5df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:27.210680 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1182/2338] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:38:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:38:27.220456 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25208c6-1475-4680-8e55-922799d16e25 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:27.221853 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-a25208c6-1475-4680-8e55-922799d16e25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:27.278803 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25208c6-1475-4680-8e55-922799d16e25 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:27.282387 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0fe905-4792-490b-a54e-83143eea45c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:27.282974 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0fe905-4792-490b-a54e-83143eea45c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:27.283250 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0fe905-4792-490b-a54e-83143eea45c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:27.283519 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0fe905-4792-490b-a54e-83143eea45c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:27.292434 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0fe905-4792-490b-a54e-83143eea45c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:27.293850 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0fe905-4792-490b-a54e-83143eea45c6 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:27.303314 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1157/2339] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:38:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:38:27.568290 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408930c-159e-4a86-b950-530f481d5df6 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:27.569759 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-6408930c-159e-4a86-b950-530f481d5df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:27.582216 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408930c-159e-4a86-b950-530f481d5df6 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:27.587450 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12459981-e450-46f4-869c-d9f91e4b85a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:27.588114 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12459981-e450-46f4-869c-d9f91e4b85a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:27.588390 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12459981-e450-46f4-869c-d9f91e4b85a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:27.588684 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12459981-e450-46f4-869c-d9f91e4b85a8 admin admin] PATH_INFO: `/v3/projects/589d9929a2144458951bc1ea280e26e0` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:27.589824 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12459981-e450-46f4-869c-d9f91e4b85a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=589d9929a2144458951bc1ea280e26e0)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:27.591761 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12459981-e450-46f4-869c-d9f91e4b85a8 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:27.606567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:27.643019 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-12459981-e450-46f4-869c-d9f91e4b85a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '589d9929a2144458951bc1ea280e26e0', 'request_id': 'req-12459981-e450-46f4-869c-d9f91e4b85a8'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:27.649112 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-12459981-e450-46f4-869c-d9f91e4b85a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '589d9929a2144458951bc1ea280e26e0', 'request_id': 'req-12459981-e450-46f4-869c-d9f91e4b85a8'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:27.656161 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-12459981-e450-46f4-869c-d9f91e4b85a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '589d9929a2144458951bc1ea280e26e0', 'request_id': 'req-12459981-e450-46f4-869c-d9f91e4b85a8'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:27.666483 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-12459981-e450-46f4-869c-d9f91e4b85a8 admin admin] The token cache is being invalidate because project 589d9929a2144458951bc1ea280e26e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:38:27.666794 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-12459981-e450-46f4-869c-d9f91e4b85a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:27.667420 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-12459981-e450-46f4-869c-d9f91e4b85a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:27.668866 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1183/2340] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:38:27 2022] DELETE /identity/v3/projects/589d9929a2144458951bc1ea280e26e0 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:38:27.672851 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0fe905-4792-490b-a54e-83143eea45c6 service neutron] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:27.683464 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0fe905-4792-490b-a54e-83143eea45c6 service neutron] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:27.688066 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:27.743932 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce8361-4693-40ae-b134-da88fa5e3b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:27.744569 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce8361-4693-40ae-b134-da88fa5e3b2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:27.744849 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce8361-4693-40ae-b134-da88fa5e3b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:27.745119 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce8361-4693-40ae-b134-da88fa5e3b2d admin admin] PATH_INFO: `/v3/users/f3c9951be9dc4a41a76280845b345e6f` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:27.746269 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce8361-4693-40ae-b134-da88fa5e3b2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3c9951be9dc4a41a76280845b345e6f)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:27.748173 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce8361-4693-40ae-b134-da88fa5e3b2d admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:27.797654 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-5bce8361-4693-40ae-b134-da88fa5e3b2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3c9951be9dc4a41a76280845b345e6f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:27.798321 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-5bce8361-4693-40ae-b134-da88fa5e3b2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3c9951be9dc4a41a76280845b345e6f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:27.798953 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-5bce8361-4693-40ae-b134-da88fa5e3b2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3c9951be9dc4a41a76280845b345e6f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:27.807455 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-5bce8361-4693-40ae-b134-da88fa5e3b2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3c9951be9dc4a41a76280845b345e6f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:27.819631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-5bce8361-4693-40ae-b134-da88fa5e3b2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3c9951be9dc4a41a76280845b345e6f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:27.833425 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1158/2341] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:38:27 2022] DELETE /identity/v3/users/f3c9951be9dc4a41a76280845b345e6f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:38:28.160704 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12459981-e450-46f4-869c-d9f91e4b85a8 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:28.162796 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-12459981-e450-46f4-869c-d9f91e4b85a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:28.176753 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12459981-e450-46f4-869c-d9f91e4b85a8 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:28.264389 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003474bb-56ce-45a8-8194-c61c01078867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:28.265098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003474bb-56ce-45a8-8194-c61c01078867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:28.265409 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003474bb-56ce-45a8-8194-c61c01078867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:28.265727 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003474bb-56ce-45a8-8194-c61c01078867 admin admin] PATH_INFO: `/v3/projects/64575736030f4f219dd8444e7bf3378b` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:28.267059 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003474bb-56ce-45a8-8194-c61c01078867 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64575736030f4f219dd8444e7bf3378b)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:28.269036 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003474bb-56ce-45a8-8194-c61c01078867 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:28.321168 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-003474bb-56ce-45a8-8194-c61c01078867 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64575736030f4f219dd8444e7bf3378b', 'request_id': 'req-003474bb-56ce-45a8-8194-c61c01078867'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:28.327859 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-003474bb-56ce-45a8-8194-c61c01078867 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64575736030f4f219dd8444e7bf3378b', 'request_id': 'req-003474bb-56ce-45a8-8194-c61c01078867'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:28.336052 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-003474bb-56ce-45a8-8194-c61c01078867 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64575736030f4f219dd8444e7bf3378b', 'request_id': 'req-003474bb-56ce-45a8-8194-c61c01078867'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:28.348035 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-003474bb-56ce-45a8-8194-c61c01078867 admin admin] The token cache is being invalidate because project 64575736030f4f219dd8444e7bf3378b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:38:28.349059 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-003474bb-56ce-45a8-8194-c61c01078867 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:28.349729 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-003474bb-56ce-45a8-8194-c61c01078867 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:28.354977 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1184/2342] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:38:28 2022] DELETE /identity/v3/projects/64575736030f4f219dd8444e7bf3378b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:38:28.356842 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7640343-38f6-4983-8e94-84aac69af86e None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:28.357138 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7640343-38f6-4983-8e94-84aac69af86e None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:28.357443 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7640343-38f6-4983-8e94-84aac69af86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:28.381672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-d7640343-38f6-4983-8e94-84aac69af86e None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:38:28.382930 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-d7640343-38f6-4983-8e94-84aac69af86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:28.448986 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1159/2343] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:38:28.456816 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e63bb-a763-4a69-8247-73b4cadc9677 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:28.457105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e63bb-a763-4a69-8247-73b4cadc9677 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:28.457389 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e63bb-a763-4a69-8247-73b4cadc9677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:28.478834 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-0f1e63bb-a763-4a69-8247-73b4cadc9677 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:38:28.480245 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-0f1e63bb-a763-4a69-8247-73b4cadc9677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:28.488979 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1185/2344] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:38:28.493547 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7640343-38f6-4983-8e94-84aac69af86e None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:28.503297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7640343-38f6-4983-8e94-84aac69af86e None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:28.508410 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44edb0a-dada-4429-99b3-cdc11b1ef554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:28.509005 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44edb0a-dada-4429-99b3-cdc11b1ef554 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:28.509285 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44edb0a-dada-4429-99b3-cdc11b1ef554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:28.509559 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44edb0a-dada-4429-99b3-cdc11b1ef554 admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:28.510240 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44edb0a-dada-4429-99b3-cdc11b1ef554 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:38:28.510545 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44edb0a-dada-4429-99b3-cdc11b1ef554 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:28.512156 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44edb0a-dada-4429-99b3-cdc11b1ef554 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:28.524764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1160/2345] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:38:28.529092 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e63bb-a763-4a69-8247-73b4cadc9677 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:28.538320 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e63bb-a763-4a69-8247-73b4cadc9677 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:28.543120 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a680576-eb03-4c3b-85b6-08bdae4c669e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:28.543743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a680576-eb03-4c3b-85b6-08bdae4c669e admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:28.544047 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a680576-eb03-4c3b-85b6-08bdae4c669e admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:28.544314 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a680576-eb03-4c3b-85b6-08bdae4c669e admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:28.544946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a680576-eb03-4c3b-85b6-08bdae4c669e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:28.546850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a680576-eb03-4c3b-85b6-08bdae4c669e admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:28.574561 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1186/2346] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:38:28 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:38:28.578749 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44edb0a-dada-4429-99b3-cdc11b1ef554 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:28.589812 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44edb0a-dada-4429-99b3-cdc11b1ef554 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:28.605976 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:28.653608 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaea831-5dd4-402d-83b4-f14ee2a39588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:28.654238 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaea831-5dd4-402d-83b4-f14ee2a39588 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:28.654515 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaea831-5dd4-402d-83b4-f14ee2a39588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:28.654784 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaea831-5dd4-402d-83b4-f14ee2a39588 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:28.655399 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaea831-5dd4-402d-83b4-f14ee2a39588 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:28.657311 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaea831-5dd4-402d-83b4-f14ee2a39588 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:28.680929 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1161/2347] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:38:28 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:38:28.682256 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a680576-eb03-4c3b-85b6-08bdae4c669e admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:28.687586 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:28.694128 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a680576-eb03-4c3b-85b6-08bdae4c669e admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:28.700159 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ab13a-c345-4613-9164-1849d5537586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:28.700688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ab13a-c345-4613-9164-1849d5537586 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:28.700960 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ab13a-c345-4613-9164-1849d5537586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:28.701223 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ab13a-c345-4613-9164-1849d5537586 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:28.701846 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230ab13a-c345-4613-9164-1849d5537586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:28.703445 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230ab13a-c345-4613-9164-1849d5537586 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:28.714636 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1187/2348] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:38:28.718584 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaea831-5dd4-402d-83b4-f14ee2a39588 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:28.728572 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaea831-5dd4-402d-83b4-f14ee2a39588 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:28.733432 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba7c6d-0d03-4c5f-bda1-249bc42087af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:28.734042 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba7c6d-0d03-4c5f-bda1-249bc42087af admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:28.734327 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba7c6d-0d03-4c5f-bda1-249bc42087af admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:28.734605 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba7c6d-0d03-4c5f-bda1-249bc42087af admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:28.735244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba7c6d-0d03-4c5f-bda1-249bc42087af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:28.736920 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba7c6d-0d03-4c5f-bda1-249bc42087af admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:28.747329 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1162/2349] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:38:28.751132 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ab13a-c345-4613-9164-1849d5537586 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:28.761361 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ab13a-c345-4613-9164-1849d5537586 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:28.765882 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02073268-92f8-4533-84f1-ffac85ca2dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:28.766554 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02073268-92f8-4533-84f1-ffac85ca2dcf admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:28.766839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02073268-92f8-4533-84f1-ffac85ca2dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:28.767110 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02073268-92f8-4533-84f1-ffac85ca2dcf admin admin] PATH_INFO: `/v3/projects/41f89553e9b24a3caa03c28443d08063/users/d553e99317a64e10a51937eabfe82763/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:28.782247 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02073268-92f8-4533-84f1-ffac85ca2dcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=41f89553e9b24a3caa03c28443d08063, user_id=d553e99317a64e10a51937eabfe82763, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:28.785102 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02073268-92f8-4533-84f1-ffac85ca2dcf admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:28.799391 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1188/2350] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:38:28 2022] PUT /identity/v3/projects/41f89553e9b24a3caa03c28443d08063/users/d553e99317a64e10a51937eabfe82763/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:38:28.809462 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edab57-a398-4140-815e-368dfaaf9fce None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:28.809748 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edab57-a398-4140-815e-368dfaaf9fce None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:28.810095 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edab57-a398-4140-815e-368dfaaf9fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:28.838757 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-88edab57-a398-4140-815e-368dfaaf9fce None None] MFA Rules not processed for user `d553e99317a64e10a51937eabfe82763`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:38:28.840004 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-88edab57-a398-4140-815e-368dfaaf9fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:28.903739 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1163/2351] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:38:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:38:28.908262 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02073268-92f8-4533-84f1-ffac85ca2dcf admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:28.969161 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02073268-92f8-4533-84f1-ffac85ca2dcf admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:28.974008 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f758e8f-493f-4202-bdb0-e9b4e97822ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:28.974632 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f758e8f-493f-4202-bdb0-e9b4e97822ac admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:28.974916 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f758e8f-493f-4202-bdb0-e9b4e97822ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:28.975185 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f758e8f-493f-4202-bdb0-e9b4e97822ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:28.975805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f758e8f-493f-4202-bdb0-e9b4e97822ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:28.977679 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f758e8f-493f-4202-bdb0-e9b4e97822ac admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:29.004153 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1189/2352] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:38:28 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:38:29.008869 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edab57-a398-4140-815e-368dfaaf9fce None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:29.019945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edab57-a398-4140-815e-368dfaaf9fce None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:29.076131 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a562a6-335d-4443-be2e-0fbc89b96a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:29.076737 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a562a6-335d-4443-be2e-0fbc89b96a4d admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:29.077016 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a562a6-335d-4443-be2e-0fbc89b96a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:29.077289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a562a6-335d-4443-be2e-0fbc89b96a4d admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:29.077932 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a562a6-335d-4443-be2e-0fbc89b96a4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:29.079885 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a562a6-335d-4443-be2e-0fbc89b96a4d admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:29.100465 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1164/2353] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:38:29 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:38:29.104492 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f758e8f-493f-4202-bdb0-e9b4e97822ac admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:29.115472 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f758e8f-493f-4202-bdb0-e9b4e97822ac admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:29.120002 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681a910-32ac-4811-b3a5-d40b4d375841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:29.120591 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681a910-32ac-4811-b3a5-d40b4d375841 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:29.120870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681a910-32ac-4811-b3a5-d40b4d375841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:29.121143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681a910-32ac-4811-b3a5-d40b4d375841 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:29.121777 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6681a910-32ac-4811-b3a5-d40b4d375841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:29.123385 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6681a910-32ac-4811-b3a5-d40b4d375841 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:29.134266 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1190/2354] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:38:29.138244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a562a6-335d-4443-be2e-0fbc89b96a4d admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:29.150117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a562a6-335d-4443-be2e-0fbc89b96a4d admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:29.155040 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6419d5c6-cd20-4610-b995-93b6bfc34197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:29.155638 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6419d5c6-cd20-4610-b995-93b6bfc34197 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:29.155918 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6419d5c6-cd20-4610-b995-93b6bfc34197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:29.156195 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6419d5c6-cd20-4610-b995-93b6bfc34197 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:29.156834 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6419d5c6-cd20-4610-b995-93b6bfc34197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:29.158463 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6419d5c6-cd20-4610-b995-93b6bfc34197 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:29.168461 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1165/2355] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:38:29.172859 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681a910-32ac-4811-b3a5-d40b4d375841 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:29.184268 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681a910-32ac-4811-b3a5-d40b4d375841 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:29.189073 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94bab25-972c-407f-aa7f-cc37b052a12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:29.189729 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bab25-972c-407f-aa7f-cc37b052a12a admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:29.190006 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bab25-972c-407f-aa7f-cc37b052a12a admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:29.190312 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bab25-972c-407f-aa7f-cc37b052a12a admin admin] PATH_INFO: `/v3/projects/f85b64c96adc465ab15fe4684e439a1c/users/bd31d13e79d647e388b94248f6f1fc5f/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:29.205472 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94bab25-972c-407f-aa7f-cc37b052a12a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f85b64c96adc465ab15fe4684e439a1c, user_id=bd31d13e79d647e388b94248f6f1fc5f, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:29.208149 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94bab25-972c-407f-aa7f-cc37b052a12a admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:29.220590 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1191/2356] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:38:29 2022] PUT /identity/v3/projects/f85b64c96adc465ab15fe4684e439a1c/users/bd31d13e79d647e388b94248f6f1fc5f/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:38:29.232682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fd69e-f072-4d23-8d26-36f12f4167ad None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:29.232972 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fd69e-f072-4d23-8d26-36f12f4167ad None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:29.233250 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fd69e-f072-4d23-8d26-36f12f4167ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:29.240994 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:29.265903 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-054fd69e-f072-4d23-8d26-36f12f4167ad None None] MFA Rules not processed for user `bd31d13e79d647e388b94248f6f1fc5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:38:29.267187 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-054fd69e-f072-4d23-8d26-36f12f4167ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:29.330428 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1166/2357] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:38:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:38:29.335039 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94bab25-972c-407f-aa7f-cc37b052a12a admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:29.398568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94bab25-972c-407f-aa7f-cc37b052a12a admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:29.403149 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf226a-0406-4e15-814f-6034e6d697cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:29.403743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf226a-0406-4e15-814f-6034e6d697cf admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:29.404017 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf226a-0406-4e15-814f-6034e6d697cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:29.404286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf226a-0406-4e15-814f-6034e6d697cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:29.404905 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bf226a-0406-4e15-814f-6034e6d697cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:29.406801 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bf226a-0406-4e15-814f-6034e6d697cf admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:29.438316 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1192/2358] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:38:29 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:38:29.439382 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054fd69e-f072-4d23-8d26-36f12f4167ad None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:29.451310 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054fd69e-f072-4d23-8d26-36f12f4167ad None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:29.514995 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ea895f-14ce-4ff9-a645-2cb467e3aa2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:29.515601 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea895f-14ce-4ff9-a645-2cb467e3aa2d admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:29.515878 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea895f-14ce-4ff9-a645-2cb467e3aa2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:29.516143 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea895f-14ce-4ff9-a645-2cb467e3aa2d admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:29.516770 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ea895f-14ce-4ff9-a645-2cb467e3aa2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:29.518788 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ea895f-14ce-4ff9-a645-2cb467e3aa2d admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:29.539699 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1167/2359] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:38:29 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:38:29.543605 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf226a-0406-4e15-814f-6034e6d697cf admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:29.549101 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:29.555840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf226a-0406-4e15-814f-6034e6d697cf admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:29.560577 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac3723f-6e6f-4c22-b3be-55d8eadeeef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:29.561162 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3723f-6e6f-4c22-b3be-55d8eadeeef0 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:29.561435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3723f-6e6f-4c22-b3be-55d8eadeeef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:29.561708 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3723f-6e6f-4c22-b3be-55d8eadeeef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:29.562370 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac3723f-6e6f-4c22-b3be-55d8eadeeef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:29.563933 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac3723f-6e6f-4c22-b3be-55d8eadeeef0 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:29.574231 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1193/2360] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:38:29.578218 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ea895f-14ce-4ff9-a645-2cb467e3aa2d admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:29.588787 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ea895f-14ce-4ff9-a645-2cb467e3aa2d admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:29.593443 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938896a-788c-4c96-8aa8-e3ab9109ed6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:29.594128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938896a-788c-4c96-8aa8-e3ab9109ed6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:29.594408 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938896a-788c-4c96-8aa8-e3ab9109ed6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:29.594767 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938896a-788c-4c96-8aa8-e3ab9109ed6e admin admin] PATH_INFO: `/v3/projects/72480d5f31cd44feb753c033748ed7d7/users/92aedbf1d4754487b61aa4da2baf0cf9/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:29.610931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8938896a-788c-4c96-8aa8-e3ab9109ed6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=72480d5f31cd44feb753c033748ed7d7, user_id=92aedbf1d4754487b61aa4da2baf0cf9, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:29.613763 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8938896a-788c-4c96-8aa8-e3ab9109ed6e admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:29.626386 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1168/2361] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:38:29 2022] PUT /identity/v3/projects/72480d5f31cd44feb753c033748ed7d7/users/92aedbf1d4754487b61aa4da2baf0cf9/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:38:29.630344 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac3723f-6e6f-4c22-b3be-55d8eadeeef0 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:29.640336 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac3723f-6e6f-4c22-b3be-55d8eadeeef0 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:29.696899 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d9aa0-2ae5-458d-9dfe-76c684fc7641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:29.697731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d9aa0-2ae5-458d-9dfe-76c684fc7641 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:29.698086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d9aa0-2ae5-458d-9dfe-76c684fc7641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:29.698368 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d9aa0-2ae5-458d-9dfe-76c684fc7641 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:29.698991 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d9aa0-2ae5-458d-9dfe-76c684fc7641 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:29.700571 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d9aa0-2ae5-458d-9dfe-76c684fc7641 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:29.710858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1194/2362] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:38:29.715151 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938896a-788c-4c96-8aa8-e3ab9109ed6e admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:29.725869 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938896a-788c-4c96-8aa8-e3ab9109ed6e admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:29.730683 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199fdb8b-3b9c-4324-9571-66e23a70e4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:29.731312 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fdb8b-3b9c-4324-9571-66e23a70e4c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:29.731591 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fdb8b-3b9c-4324-9571-66e23a70e4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:29.731860 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fdb8b-3b9c-4324-9571-66e23a70e4c8 admin admin] PATH_INFO: `/v3/projects/72480d5f31cd44feb753c033748ed7d7/users/92aedbf1d4754487b61aa4da2baf0cf9/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:29.733722 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199fdb8b-3b9c-4324-9571-66e23a70e4c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72480d5f31cd44feb753c033748ed7d7, user_id=92aedbf1d4754487b61aa4da2baf0cf9, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:29.736343 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199fdb8b-3b9c-4324-9571-66e23a70e4c8 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:29.746314 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1169/2363] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:38:29 2022] PUT /identity/v3/projects/72480d5f31cd44feb753c033748ed7d7/users/92aedbf1d4754487b61aa4da2baf0cf9/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:38:29.756367 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a7457-7141-482d-ac8a-3a47be07d19a None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:29.756655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a7457-7141-482d-ac8a-3a47be07d19a None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:29.756933 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a7457-7141-482d-ac8a-3a47be07d19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:29.784791 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-599a7457-7141-482d-ac8a-3a47be07d19a None None] MFA Rules not processed for user `92aedbf1d4754487b61aa4da2baf0cf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:38:29.786550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-599a7457-7141-482d-ac8a-3a47be07d19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:29.854144 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1195/2364] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:38:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:38:29.865372 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199fdb8b-3b9c-4324-9571-66e23a70e4c8 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:29.867124 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-199fdb8b-3b9c-4324-9571-66e23a70e4c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:29.913425 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:29.924836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199fdb8b-3b9c-4324-9571-66e23a70e4c8 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:29.928545 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b650d435-6825-4e03-b810-1f8f048c9677 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:29.929190 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650d435-6825-4e03-b810-1f8f048c9677 service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:29.929466 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650d435-6825-4e03-b810-1f8f048c9677 service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:29.929736 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650d435-6825-4e03-b810-1f8f048c9677 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:29.938819 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b650d435-6825-4e03-b810-1f8f048c9677 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:29.940260 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b650d435-6825-4e03-b810-1f8f048c9677 service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:29.949759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1170/2365] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:38:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:38:29.991629 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a7457-7141-482d-ac8a-3a47be07d19a None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:29.993000 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-599a7457-7141-482d-ac8a-3a47be07d19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:30.007764 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a7457-7141-482d-ac8a-3a47be07d19a None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:30.011479 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcbb2c-3adb-4059-b964-ca0be060d1e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:30.012074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcbb2c-3adb-4059-b964-ca0be060d1e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:30.012350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcbb2c-3adb-4059-b964-ca0be060d1e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:30.012650 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcbb2c-3adb-4059-b964-ca0be060d1e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:30.069691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbcbb2c-3adb-4059-b964-ca0be060d1e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:30.071052 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbcbb2c-3adb-4059-b964-ca0be060d1e3 service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:30.079764 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1196/2366] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:38:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:38:32.999424 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b650d435-6825-4e03-b810-1f8f048c9677 service cinder] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:33.060893 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b650d435-6825-4e03-b810-1f8f048c9677 service cinder] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:33.065473 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412993cb-4b00-4744-bd86-38c1ab019caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:33.066145 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412993cb-4b00-4744-bd86-38c1ab019caa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:33.066429 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412993cb-4b00-4744-bd86-38c1ab019caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:33.066700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412993cb-4b00-4744-bd86-38c1ab019caa admin admin] PATH_INFO: `/v3/users/d553e99317a64e10a51937eabfe82763` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:33.068011 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412993cb-4b00-4744-bd86-38c1ab019caa admin admin] RBAC: Authorizing `identity:delete_user(user_id=d553e99317a64e10a51937eabfe82763)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:33.070398 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412993cb-4b00-4744-bd86-38c1ab019caa admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:33.117535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-412993cb-4b00-4744-bd86-38c1ab019caa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd553e99317a64e10a51937eabfe82763'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:33.118241 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-412993cb-4b00-4744-bd86-38c1ab019caa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd553e99317a64e10a51937eabfe82763'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:33.118845 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-412993cb-4b00-4744-bd86-38c1ab019caa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd553e99317a64e10a51937eabfe82763'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:33.125617 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-412993cb-4b00-4744-bd86-38c1ab019caa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd553e99317a64e10a51937eabfe82763'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:33.137442 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-412993cb-4b00-4744-bd86-38c1ab019caa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd553e99317a64e10a51937eabfe82763'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:33.150112 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1171/2367] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:38:32 2022] DELETE /identity/v3/users/d553e99317a64e10a51937eabfe82763 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:38:33.154581 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa582f-2fd5-4687-b088-f88cec57747c None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:33.155120 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa582f-2fd5-4687-b088-f88cec57747c None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:33.155120 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa582f-2fd5-4687-b088-f88cec57747c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:33.175029 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-58fa582f-2fd5-4687-b088-f88cec57747c None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:38:33.176229 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-58fa582f-2fd5-4687-b088-f88cec57747c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:33.181731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:33.235293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1197/2368] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:38:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:38:33.245419 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412993cb-4b00-4744-bd86-38c1ab019caa admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:33.246859 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-412993cb-4b00-4744-bd86-38c1ab019caa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:33.303520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412993cb-4b00-4744-bd86-38c1ab019caa admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:33.307150 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5368910-faf6-438e-814d-0eaa534884e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:33.307732 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5368910-faf6-438e-814d-0eaa534884e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:33.308011 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5368910-faf6-438e-814d-0eaa534884e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:33.308285 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5368910-faf6-438e-814d-0eaa534884e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:33.317285 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5368910-faf6-438e-814d-0eaa534884e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:33.318704 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5368910-faf6-438e-814d-0eaa534884e2 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:33.329064 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1172/2369] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:38:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:38:33.592002 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa582f-2fd5-4687-b088-f88cec57747c None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:33.593406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-58fa582f-2fd5-4687-b088-f88cec57747c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:33.606925 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa582f-2fd5-4687-b088-f88cec57747c None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:33.611542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b737b-4c35-4cda-836d-84201d924e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:33.612160 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b737b-4c35-4cda-836d-84201d924e50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:33.612436 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b737b-4c35-4cda-836d-84201d924e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:33.612700 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b737b-4c35-4cda-836d-84201d924e50 admin admin] PATH_INFO: `/v3/projects/41f89553e9b24a3caa03c28443d08063` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:33.613808 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b737b-4c35-4cda-836d-84201d924e50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41f89553e9b24a3caa03c28443d08063)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:33.616364 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b737b-4c35-4cda-836d-84201d924e50 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:33.662734 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-696b737b-4c35-4cda-836d-84201d924e50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41f89553e9b24a3caa03c28443d08063', 'request_id': 'req-696b737b-4c35-4cda-836d-84201d924e50'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:33.668792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-696b737b-4c35-4cda-836d-84201d924e50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41f89553e9b24a3caa03c28443d08063', 'request_id': 'req-696b737b-4c35-4cda-836d-84201d924e50'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:33.676082 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-696b737b-4c35-4cda-836d-84201d924e50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41f89553e9b24a3caa03c28443d08063', 'request_id': 'req-696b737b-4c35-4cda-836d-84201d924e50'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:33.687181 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-696b737b-4c35-4cda-836d-84201d924e50 admin admin] The token cache is being invalidate because project 41f89553e9b24a3caa03c28443d08063 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:38:33.687573 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-696b737b-4c35-4cda-836d-84201d924e50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:33.688273 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-696b737b-4c35-4cda-836d-84201d924e50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:33.689764 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1198/2370] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:38:33 2022] DELETE /identity/v3/projects/41f89553e9b24a3caa03c28443d08063 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:38:33.694340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5368910-faf6-438e-814d-0eaa534884e2 service neutron] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:33.704343 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5368910-faf6-438e-814d-0eaa534884e2 service neutron] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:33.709080 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:33.763772 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f804b0a2-d613-49ba-af45-79470c27b2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:33.764479 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f804b0a2-d613-49ba-af45-79470c27b2a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:33.765290 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f804b0a2-d613-49ba-af45-79470c27b2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:33.765290 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f804b0a2-d613-49ba-af45-79470c27b2a4 admin admin] PATH_INFO: `/v3/users/bd31d13e79d647e388b94248f6f1fc5f` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:33.766451 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f804b0a2-d613-49ba-af45-79470c27b2a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd31d13e79d647e388b94248f6f1fc5f)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:33.768393 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f804b0a2-d613-49ba-af45-79470c27b2a4 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:33.817307 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-f804b0a2-d613-49ba-af45-79470c27b2a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd31d13e79d647e388b94248f6f1fc5f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:33.817981 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-f804b0a2-d613-49ba-af45-79470c27b2a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd31d13e79d647e388b94248f6f1fc5f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:33.818671 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-f804b0a2-d613-49ba-af45-79470c27b2a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd31d13e79d647e388b94248f6f1fc5f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:33.825755 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-f804b0a2-d613-49ba-af45-79470c27b2a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd31d13e79d647e388b94248f6f1fc5f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:33.837773 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-f804b0a2-d613-49ba-af45-79470c27b2a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd31d13e79d647e388b94248f6f1fc5f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:33.852040 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1173/2371] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:38:33 2022] DELETE /identity/v3/users/bd31d13e79d647e388b94248f6f1fc5f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:38:34.188568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b737b-4c35-4cda-836d-84201d924e50 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:34.190207 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-696b737b-4c35-4cda-836d-84201d924e50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:34.206419 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b737b-4c35-4cda-836d-84201d924e50 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:34.281863 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a81a5-0644-4b79-9970-5163c4913022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:34.282531 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a81a5-0644-4b79-9970-5163c4913022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:34.282813 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a81a5-0644-4b79-9970-5163c4913022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:34.283089 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a81a5-0644-4b79-9970-5163c4913022 admin admin] PATH_INFO: `/v3/projects/f85b64c96adc465ab15fe4684e439a1c` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:34.284204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a81a5-0644-4b79-9970-5163c4913022 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f85b64c96adc465ab15fe4684e439a1c)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:34.286737 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a81a5-0644-4b79-9970-5163c4913022 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:34.323003 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:34.332264 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-6b5a81a5-0644-4b79-9970-5163c4913022 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f85b64c96adc465ab15fe4684e439a1c', 'request_id': 'req-6b5a81a5-0644-4b79-9970-5163c4913022'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:34.339015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-6b5a81a5-0644-4b79-9970-5163c4913022 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f85b64c96adc465ab15fe4684e439a1c', 'request_id': 'req-6b5a81a5-0644-4b79-9970-5163c4913022'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:34.346372 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-6b5a81a5-0644-4b79-9970-5163c4913022 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f85b64c96adc465ab15fe4684e439a1c', 'request_id': 'req-6b5a81a5-0644-4b79-9970-5163c4913022'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:34.357588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-6b5a81a5-0644-4b79-9970-5163c4913022 admin admin] The token cache is being invalidate because project f85b64c96adc465ab15fe4684e439a1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:38:34.357898 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-6b5a81a5-0644-4b79-9970-5163c4913022 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:34.358540 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-6b5a81a5-0644-4b79-9970-5163c4913022 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:34.359993 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1199/2372] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:38:34 2022] DELETE /identity/v3/projects/f85b64c96adc465ab15fe4684e439a1c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:38:34.364160 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f804b0a2-d613-49ba-af45-79470c27b2a4 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:34.365579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-f804b0a2-d613-49ba-af45-79470c27b2a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:34.378638 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f804b0a2-d613-49ba-af45-79470c27b2a4 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:34.444630 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568e910-2e38-4dcd-ae9f-9047bd2eb016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:34.445308 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568e910-2e38-4dcd-ae9f-9047bd2eb016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:34.445589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568e910-2e38-4dcd-ae9f-9047bd2eb016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:34.446247 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568e910-2e38-4dcd-ae9f-9047bd2eb016 admin admin] PATH_INFO: `/v3/users/92aedbf1d4754487b61aa4da2baf0cf9` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:34.447226 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568e910-2e38-4dcd-ae9f-9047bd2eb016 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92aedbf1d4754487b61aa4da2baf0cf9)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:34.449067 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568e910-2e38-4dcd-ae9f-9047bd2eb016 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:34.499237 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:34.515144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-0568e910-2e38-4dcd-ae9f-9047bd2eb016 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92aedbf1d4754487b61aa4da2baf0cf9'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:34.515832 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-0568e910-2e38-4dcd-ae9f-9047bd2eb016 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92aedbf1d4754487b61aa4da2baf0cf9'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:34.516513 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-0568e910-2e38-4dcd-ae9f-9047bd2eb016 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92aedbf1d4754487b61aa4da2baf0cf9'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:34.524832 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-0568e910-2e38-4dcd-ae9f-9047bd2eb016 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92aedbf1d4754487b61aa4da2baf0cf9'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:34.538470 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-0568e910-2e38-4dcd-ae9f-9047bd2eb016 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92aedbf1d4754487b61aa4da2baf0cf9'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:34.551390 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1174/2373] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:38:34 2022] DELETE /identity/v3/users/92aedbf1d4754487b61aa4da2baf0cf9 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:38:34.823619 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a81a5-0644-4b79-9970-5163c4913022 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:34.825046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-6b5a81a5-0644-4b79-9970-5163c4913022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:34.838270 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a81a5-0644-4b79-9970-5163c4913022 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:34.894594 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6585f9-aab7-4df4-b7fa-57030a6dd476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:34.895224 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6585f9-aab7-4df4-b7fa-57030a6dd476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:34.896015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6585f9-aab7-4df4-b7fa-57030a6dd476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:34.896290 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6585f9-aab7-4df4-b7fa-57030a6dd476 admin admin] PATH_INFO: `/v3/projects/72480d5f31cd44feb753c033748ed7d7` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:34.897411 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6585f9-aab7-4df4-b7fa-57030a6dd476 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72480d5f31cd44feb753c033748ed7d7)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:34.899336 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6585f9-aab7-4df4-b7fa-57030a6dd476 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:34.944745 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-9c6585f9-aab7-4df4-b7fa-57030a6dd476 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72480d5f31cd44feb753c033748ed7d7', 'request_id': 'req-9c6585f9-aab7-4df4-b7fa-57030a6dd476'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:34.950755 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-9c6585f9-aab7-4df4-b7fa-57030a6dd476 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72480d5f31cd44feb753c033748ed7d7', 'request_id': 'req-9c6585f9-aab7-4df4-b7fa-57030a6dd476'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:34.958850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-9c6585f9-aab7-4df4-b7fa-57030a6dd476 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72480d5f31cd44feb753c033748ed7d7', 'request_id': 'req-9c6585f9-aab7-4df4-b7fa-57030a6dd476'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:34.969056 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-9c6585f9-aab7-4df4-b7fa-57030a6dd476 admin admin] The token cache is being invalidate because project 72480d5f31cd44feb753c033748ed7d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:38:34.969334 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-9c6585f9-aab7-4df4-b7fa-57030a6dd476 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:34.969940 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-9c6585f9-aab7-4df4-b7fa-57030a6dd476 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:34.971405 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1200/2374] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:38:34 2022] DELETE /identity/v3/projects/72480d5f31cd44feb753c033748ed7d7 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:38:34.978242 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba3d26-6dd8-4de0-be3e-4d0e4e050ee5 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:34.978524 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba3d26-6dd8-4de0-be3e-4d0e4e050ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:34.978806 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba3d26-6dd8-4de0-be3e-4d0e4e050ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:34.998004 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-31ba3d26-6dd8-4de0-be3e-4d0e4e050ee5 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:38:34.999219 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-31ba3d26-6dd8-4de0-be3e-4d0e4e050ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:35.062287 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1175/2375] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:38:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:38:35.070334 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ea09a-eaeb-4995-9d50-21061ce08626 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:35.070624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ea09a-eaeb-4995-9d50-21061ce08626 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:35.070907 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ea09a-eaeb-4995-9d50-21061ce08626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:35.093211 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-a39ea09a-eaeb-4995-9d50-21061ce08626 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:38:35.094460 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-a39ea09a-eaeb-4995-9d50-21061ce08626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:35.103171 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1201/2376] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:38:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:38:35.107686 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba3d26-6dd8-4de0-be3e-4d0e4e050ee5 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:35.117284 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba3d26-6dd8-4de0-be3e-4d0e4e050ee5 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:35.121881 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1973962b-34b0-4220-a0c6-7b1b38d3e6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:35.122497 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1973962b-34b0-4220-a0c6-7b1b38d3e6fd admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:35.122774 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1973962b-34b0-4220-a0c6-7b1b38d3e6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:35.123040 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1973962b-34b0-4220-a0c6-7b1b38d3e6fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:35.123672 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1973962b-34b0-4220-a0c6-7b1b38d3e6fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:38:35.123967 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1973962b-34b0-4220-a0c6-7b1b38d3e6fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:35.125542 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1973962b-34b0-4220-a0c6-7b1b38d3e6fd admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:35.136850 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1176/2377] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:38:35.140929 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ea09a-eaeb-4995-9d50-21061ce08626 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:35.150473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ea09a-eaeb-4995-9d50-21061ce08626 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:35.160249 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c82ba-9204-4a2b-8b46-5805a2350586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:35.160850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c82ba-9204-4a2b-8b46-5805a2350586 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:35.161177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c82ba-9204-4a2b-8b46-5805a2350586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:35.162066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c82ba-9204-4a2b-8b46-5805a2350586 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:35.162715 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c82ba-9204-4a2b-8b46-5805a2350586 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:35.164593 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c82ba-9204-4a2b-8b46-5805a2350586 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:35.191884 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1202/2378] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:38:35 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:38:35.196576 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1973962b-34b0-4220-a0c6-7b1b38d3e6fd admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:35.206685 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1973962b-34b0-4220-a0c6-7b1b38d3e6fd admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:35.263940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8e843-5b37-4cb9-883a-401cf56dae4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:35.264642 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8e843-5b37-4cb9-883a-401cf56dae4b admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:35.264917 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8e843-5b37-4cb9-883a-401cf56dae4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:35.265185 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8e843-5b37-4cb9-883a-401cf56dae4b admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:35.265860 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a8e843-5b37-4cb9-883a-401cf56dae4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:35.267789 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a8e843-5b37-4cb9-883a-401cf56dae4b admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:35.291200 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1177/2379] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:38:35 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:38:35.295232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c82ba-9204-4a2b-8b46-5805a2350586 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:35.305581 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c82ba-9204-4a2b-8b46-5805a2350586 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:35.310205 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c776dfe4-6d2a-49f2-ae52-35f342ff2a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:35.310797 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776dfe4-6d2a-49f2-ae52-35f342ff2a8b admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:35.311070 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776dfe4-6d2a-49f2-ae52-35f342ff2a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:35.311336 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776dfe4-6d2a-49f2-ae52-35f342ff2a8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:35.311971 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c776dfe4-6d2a-49f2-ae52-35f342ff2a8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:35.313534 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c776dfe4-6d2a-49f2-ae52-35f342ff2a8b admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:35.324133 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1203/2380] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:38:35.328162 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8e843-5b37-4cb9-883a-401cf56dae4b admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:35.341166 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8e843-5b37-4cb9-883a-401cf56dae4b admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:35.345798 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f532dc-3c4b-47e3-8993-a75bbe2d69d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:35.346421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f532dc-3c4b-47e3-8993-a75bbe2d69d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:35.346696 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f532dc-3c4b-47e3-8993-a75bbe2d69d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:35.346968 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f532dc-3c4b-47e3-8993-a75bbe2d69d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:35.347630 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f532dc-3c4b-47e3-8993-a75bbe2d69d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:35.349243 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f532dc-3c4b-47e3-8993-a75bbe2d69d6 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:35.360904 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1178/2381] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:38:35.364845 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c776dfe4-6d2a-49f2-ae52-35f342ff2a8b admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:35.374878 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c776dfe4-6d2a-49f2-ae52-35f342ff2a8b admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:35.379683 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ece5b5-5c2b-40d4-9efa-92895e9d1097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:35.380908 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ece5b5-5c2b-40d4-9efa-92895e9d1097 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:35.381187 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ece5b5-5c2b-40d4-9efa-92895e9d1097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:35.381528 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ece5b5-5c2b-40d4-9efa-92895e9d1097 admin admin] PATH_INFO: `/v3/projects/931de015d3ab4db780d3b45a75afa0bb/users/c5a332fa8c854c289c094795b556b100/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:35.394479 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:35.398856 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ece5b5-5c2b-40d4-9efa-92895e9d1097 admin admin] RBAC: Authorizing `identity:create_grant(project_id=931de015d3ab4db780d3b45a75afa0bb, user_id=c5a332fa8c854c289c094795b556b100, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:35.402600 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ece5b5-5c2b-40d4-9efa-92895e9d1097 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:35.416562 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1204/2382] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:38:35 2022] PUT /identity/v3/projects/931de015d3ab4db780d3b45a75afa0bb/users/c5a332fa8c854c289c094795b556b100/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:38:35.421727 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5363582-4634-4c82-80e9-d412caef9cd5 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:35.422235 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5363582-4634-4c82-80e9-d412caef9cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:35.422333 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5363582-4634-4c82-80e9-d412caef9cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:35.441863 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-c5363582-4634-4c82-80e9-d412caef9cd5 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:38:35.443793 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-c5363582-4634-4c82-80e9-d412caef9cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:35.452272 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:35.503937 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1179/2383] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:38:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:38:35.514029 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ece5b5-5c2b-40d4-9efa-92895e9d1097 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:35.515858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-d8ece5b5-5c2b-40d4-9efa-92895e9d1097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:35.578593 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ece5b5-5c2b-40d4-9efa-92895e9d1097 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:35.582209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a555495a-d491-4eb8-b75d-c944732b4bf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:35.582815 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a555495a-d491-4eb8-b75d-c944732b4bf4 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:35.583177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a555495a-d491-4eb8-b75d-c944732b4bf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:35.583950 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a555495a-d491-4eb8-b75d-c944732b4bf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:35.593322 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a555495a-d491-4eb8-b75d-c944732b4bf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:35.594857 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a555495a-d491-4eb8-b75d-c944732b4bf4 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:35.605125 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1205/2384] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:38:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:38:36.277867 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572c924-9d6c-4d36-92ac-7270533fdf06 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:36.278210 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572c924-9d6c-4d36-92ac-7270533fdf06 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:36.278484 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572c924-9d6c-4d36-92ac-7270533fdf06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:36.308172 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-6572c924-9d6c-4d36-92ac-7270533fdf06 None None] MFA Rules not processed for user `c5a332fa8c854c289c094795b556b100`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:38:36.309387 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-6572c924-9d6c-4d36-92ac-7270533fdf06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:36.391520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1180/2385] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:38:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:38:36.396625 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a555495a-d491-4eb8-b75d-c944732b4bf4 service neutron] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:36.406549 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a555495a-d491-4eb8-b75d-c944732b4bf4 service neutron] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:36.411002 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c0f57-0548-475e-bd7d-95525259657e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:36.412149 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c0f57-0548-475e-bd7d-95525259657e admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:36.412422 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c0f57-0548-475e-bd7d-95525259657e admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:36.412859 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c0f57-0548-475e-bd7d-95525259657e admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:36.413507 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c0f57-0548-475e-bd7d-95525259657e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:36.415411 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c0f57-0548-475e-bd7d-95525259657e admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:36.450779 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1206/2386] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:38:36 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:38:36.454882 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6572c924-9d6c-4d36-92ac-7270533fdf06 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:36.467176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6572c924-9d6c-4d36-92ac-7270533fdf06 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:36.532447 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab42b5-c380-4afa-ab5a-f6a5705260be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:36.533063 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab42b5-c380-4afa-ab5a-f6a5705260be admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:36.533342 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab42b5-c380-4afa-ab5a-f6a5705260be admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:36.533617 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab42b5-c380-4afa-ab5a-f6a5705260be admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:36.534308 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eab42b5-c380-4afa-ab5a-f6a5705260be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:36.537120 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eab42b5-c380-4afa-ab5a-f6a5705260be admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:36.560165 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1181/2387] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:38:36 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:38:36.564131 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c0f57-0548-475e-bd7d-95525259657e admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:36.574556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c0f57-0548-475e-bd7d-95525259657e admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:36.579250 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7bd4a5-a40b-4217-a7d4-cb3b83d7c6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:36.579897 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7bd4a5-a40b-4217-a7d4-cb3b83d7c6e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:36.580180 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7bd4a5-a40b-4217-a7d4-cb3b83d7c6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:36.580451 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7bd4a5-a40b-4217-a7d4-cb3b83d7c6e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:36.581098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7bd4a5-a40b-4217-a7d4-cb3b83d7c6e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:36.582737 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7bd4a5-a40b-4217-a7d4-cb3b83d7c6e4 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:36.595351 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1207/2388] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:38:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:38:36.605027 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab42b5-c380-4afa-ab5a-f6a5705260be admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:36.616286 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab42b5-c380-4afa-ab5a-f6a5705260be admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:36.620869 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3466ec7-811c-41d3-a23c-3beaee2eaf0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:36.621511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3466ec7-811c-41d3-a23c-3beaee2eaf0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:36.621786 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3466ec7-811c-41d3-a23c-3beaee2eaf0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:36.622108 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3466ec7-811c-41d3-a23c-3beaee2eaf0d admin admin] PATH_INFO: `/v3/projects/aeb05fa6f70c406eba3e5bff61fb334d/users/409353a112304a9792ccf810ba286070/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:36.636560 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3466ec7-811c-41d3-a23c-3beaee2eaf0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeb05fa6f70c406eba3e5bff61fb334d, user_id=409353a112304a9792ccf810ba286070, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:36.639226 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3466ec7-811c-41d3-a23c-3beaee2eaf0d admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:36.652217 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1182/2389] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:38:36 2022] PUT /identity/v3/projects/aeb05fa6f70c406eba3e5bff61fb334d/users/409353a112304a9792ccf810ba286070/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:38:36.656130 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7bd4a5-a40b-4217-a7d4-cb3b83d7c6e4 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:36.666878 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7bd4a5-a40b-4217-a7d4-cb3b83d7c6e4 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:36.723932 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16db2701-9b84-45e5-8486-3969f7bac869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:36.724556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db2701-9b84-45e5-8486-3969f7bac869 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:36.724840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db2701-9b84-45e5-8486-3969f7bac869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:36.725113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db2701-9b84-45e5-8486-3969f7bac869 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:36.725757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16db2701-9b84-45e5-8486-3969f7bac869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:36.727490 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16db2701-9b84-45e5-8486-3969f7bac869 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:36.737997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1208/2390] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:38:36 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:38:36.742189 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3466ec7-811c-41d3-a23c-3beaee2eaf0d admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:36.752328 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3466ec7-811c-41d3-a23c-3beaee2eaf0d admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:36.757457 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f39f8-f8a7-4087-88bd-17e28a818676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:36.758131 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f39f8-f8a7-4087-88bd-17e28a818676 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:36.758411 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f39f8-f8a7-4087-88bd-17e28a818676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:36.758684 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f39f8-f8a7-4087-88bd-17e28a818676 admin admin] PATH_INFO: `/v3/projects/aeb05fa6f70c406eba3e5bff61fb334d/users/409353a112304a9792ccf810ba286070/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:36.760714 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08f39f8-f8a7-4087-88bd-17e28a818676 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeb05fa6f70c406eba3e5bff61fb334d, user_id=409353a112304a9792ccf810ba286070, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:36.763381 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08f39f8-f8a7-4087-88bd-17e28a818676 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:36.766517 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:36.775846 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1183/2391] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:38:36 2022] PUT /identity/v3/projects/aeb05fa6f70c406eba3e5bff61fb334d/users/409353a112304a9792ccf810ba286070/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:38:37.416095 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111bfc8a-bf96-4f24-a7d9-8dcf4c6ef07d None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:37.416604 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111bfc8a-bf96-4f24-a7d9-8dcf4c6ef07d None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:37.416695 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111bfc8a-bf96-4f24-a7d9-8dcf4c6ef07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:37.454280 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-111bfc8a-bf96-4f24-a7d9-8dcf4c6ef07d None None] MFA Rules not processed for user `409353a112304a9792ccf810ba286070`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:38:37.456690 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-111bfc8a-bf96-4f24-a7d9-8dcf4c6ef07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:37.506900 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:37.542895 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1209/2392] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:38:37 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:38:37.553555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f39f8-f8a7-4087-88bd-17e28a818676 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:37.555383 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-d08f39f8-f8a7-4087-88bd-17e28a818676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:37.622802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f39f8-f8a7-4087-88bd-17e28a818676 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:37.627204 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49432146-0f05-44db-99ce-cebf59c0d519 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:37.627817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49432146-0f05-44db-99ce-cebf59c0d519 service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:37.628094 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49432146-0f05-44db-99ce-cebf59c0d519 service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:37.628396 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49432146-0f05-44db-99ce-cebf59c0d519 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:37.687396 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49432146-0f05-44db-99ce-cebf59c0d519 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:37.688740 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49432146-0f05-44db-99ce-cebf59c0d519 service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:37.699876 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1184/2393] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:38:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:38:39.067772 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111bfc8a-bf96-4f24-a7d9-8dcf4c6ef07d None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:39.077875 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111bfc8a-bf96-4f24-a7d9-8dcf4c6ef07d None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:39.081437 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619db0c-b9e2-470c-9bbc-6b13a9907149 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:39.082123 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619db0c-b9e2-470c-9bbc-6b13a9907149 service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:39.082411 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619db0c-b9e2-470c-9bbc-6b13a9907149 service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:39.082708 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619db0c-b9e2-470c-9bbc-6b13a9907149 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:39.092967 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f619db0c-b9e2-470c-9bbc-6b13a9907149 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:39.094414 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f619db0c-b9e2-470c-9bbc-6b13a9907149 service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:39.104412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1210/2394] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:38:39 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:38:47.695153 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=50 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:48.953100 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b866c1-be9c-4051-816d-140e782eb10f None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:48.953687 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b866c1-be9c-4051-816d-140e782eb10f None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:48.953687 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b866c1-be9c-4051-816d-140e782eb10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:48.973819 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-e0b866c1-be9c-4051-816d-140e782eb10f None None] MFA Rules not processed for user `9158557f1a2947e2815dc8d586d8b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:38:48.975185 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-e0b866c1-be9c-4051-816d-140e782eb10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:49.033344 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1185/2395] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Sep 26 10:38:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:38:49.045907 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619db0c-b9e2-470c-9bbc-6b13a9907149 service cinder] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:49.047398 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-f619db0c-b9e2-470c-9bbc-6b13a9907149 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:49.122683 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619db0c-b9e2-470c-9bbc-6b13a9907149 service cinder] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:49.127940 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e6fe5e-1cb5-42e4-bc84-5d3714e53a0b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e6be34fc93a4551a83e5b8380188a6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:49.128735 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6fe5e-1cb5-42e4-bc84-5d3714e53a0b service swift] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:49.129076 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6fe5e-1cb5-42e4-bc84-5d3714e53a0b service swift] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:49.129344 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6fe5e-1cb5-42e4-bc84-5d3714e53a0b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:49.139567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e6fe5e-1cb5-42e4-bc84-5d3714e53a0b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:49.141141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e6fe5e-1cb5-42e4-bc84-5d3714e53a0b service swift] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:49.150248 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1211/2396] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 10:38:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 10:38:50.536602 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b866c1-be9c-4051-816d-140e782eb10f None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:50.538247 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-e0b866c1-be9c-4051-816d-140e782eb10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:50.680599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b866c1-be9c-4051-816d-140e782eb10f None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:50.686936 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718f94b-c44a-4d4f-b073-854d942813fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:50.687334 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718f94b-c44a-4d4f-b073-854d942813fd service nova] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:50.687600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718f94b-c44a-4d4f-b073-854d942813fd service nova] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:50.687874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718f94b-c44a-4d4f-b073-854d942813fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:50.690083 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-7718f94b-c44a-4d4f-b073-854d942813fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:50.707389 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7718f94b-c44a-4d4f-b073-854d942813fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:50.709017 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7718f94b-c44a-4d4f-b073-854d942813fd service nova] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:50.735095 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1186/2397] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:38:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:38:52.645312 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e6fe5e-1cb5-42e4-bc84-5d3714e53a0b service swift] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:52.729867 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e6fe5e-1cb5-42e4-bc84-5d3714e53a0b service swift] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:52.736611 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f61c9-c8e5-4bc3-8efd-6693db46b92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:52.738237 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f61c9-c8e5-4bc3-8efd-6693db46b92f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:52.738729 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f61c9-c8e5-4bc3-8efd-6693db46b92f admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:52.739251 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f61c9-c8e5-4bc3-8efd-6693db46b92f admin admin] PATH_INFO: `/v3/users/c5a332fa8c854c289c094795b556b100` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:52.740588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f61c9-c8e5-4bc3-8efd-6693db46b92f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5a332fa8c854c289c094795b556b100)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:52.743695 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f61c9-c8e5-4bc3-8efd-6693db46b92f admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:52.823327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-e62f61c9-c8e5-4bc3-8efd-6693db46b92f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5a332fa8c854c289c094795b556b100'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:52.824006 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-e62f61c9-c8e5-4bc3-8efd-6693db46b92f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5a332fa8c854c289c094795b556b100'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:52.837974 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-e62f61c9-c8e5-4bc3-8efd-6693db46b92f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5a332fa8c854c289c094795b556b100'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:52.843282 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:52.854078 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-e62f61c9-c8e5-4bc3-8efd-6693db46b92f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5a332fa8c854c289c094795b556b100'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:52.858541 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-e62f61c9-c8e5-4bc3-8efd-6693db46b92f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5a332fa8c854c289c094795b556b100'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:52.877876 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1212/2398] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:38:52 2022] DELETE /identity/v3/users/c5a332fa8c854c289c094795b556b100 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:38:53.323638 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718f94b-c44a-4d4f-b073-854d942813fd service nova] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:53.337067 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718f94b-c44a-4d4f-b073-854d942813fd service nova] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:53.442109 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104e754-2479-4b45-8a40-5d69334aeaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:53.443719 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104e754-2479-4b45-8a40-5d69334aeaef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:53.444151 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104e754-2479-4b45-8a40-5d69334aeaef admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:53.445140 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104e754-2479-4b45-8a40-5d69334aeaef admin admin] PATH_INFO: `/v3/projects/931de015d3ab4db780d3b45a75afa0bb` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:53.447194 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104e754-2479-4b45-8a40-5d69334aeaef admin admin] RBAC: Authorizing `identity:delete_project(project_id=931de015d3ab4db780d3b45a75afa0bb)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:53.451005 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104e754-2479-4b45-8a40-5d69334aeaef admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:53.507476 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:53.522962 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-2104e754-2479-4b45-8a40-5d69334aeaef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '931de015d3ab4db780d3b45a75afa0bb', 'request_id': 'req-2104e754-2479-4b45-8a40-5d69334aeaef'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:53.533423 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-2104e754-2479-4b45-8a40-5d69334aeaef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '931de015d3ab4db780d3b45a75afa0bb', 'request_id': 'req-2104e754-2479-4b45-8a40-5d69334aeaef'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:53.541678 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-2104e754-2479-4b45-8a40-5d69334aeaef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '931de015d3ab4db780d3b45a75afa0bb', 'request_id': 'req-2104e754-2479-4b45-8a40-5d69334aeaef'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:53.578649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-2104e754-2479-4b45-8a40-5d69334aeaef admin admin] The token cache is being invalidate because project 931de015d3ab4db780d3b45a75afa0bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:38:53.578953 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-2104e754-2479-4b45-8a40-5d69334aeaef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:53.583339 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-2104e754-2479-4b45-8a40-5d69334aeaef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:53.588673 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1187/2399] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:38:53 2022] DELETE /identity/v3/projects/931de015d3ab4db780d3b45a75afa0bb => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:38:53.589924 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f61c9-c8e5-4bc3-8efd-6693db46b92f admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:53.591410 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-e62f61c9-c8e5-4bc3-8efd-6693db46b92f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:53.619842 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f61c9-c8e5-4bc3-8efd-6693db46b92f admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:53.710337 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19435e1-aa32-4530-ac30-22e436c3de5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:53.712071 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19435e1-aa32-4530-ac30-22e436c3de5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:53.712451 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19435e1-aa32-4530-ac30-22e436c3de5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:53.712847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19435e1-aa32-4530-ac30-22e436c3de5b admin admin] PATH_INFO: `/v3/users/409353a112304a9792ccf810ba286070` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:53.714202 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19435e1-aa32-4530-ac30-22e436c3de5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=409353a112304a9792ccf810ba286070)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:53.716355 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19435e1-aa32-4530-ac30-22e436c3de5b admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:53.804640 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f19435e1-aa32-4530-ac30-22e436c3de5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '409353a112304a9792ccf810ba286070'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:53.805686 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f19435e1-aa32-4530-ac30-22e436c3de5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '409353a112304a9792ccf810ba286070'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:53.826533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f19435e1-aa32-4530-ac30-22e436c3de5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '409353a112304a9792ccf810ba286070'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:53.859676 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f19435e1-aa32-4530-ac30-22e436c3de5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '409353a112304a9792ccf810ba286070'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:53.862055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f19435e1-aa32-4530-ac30-22e436c3de5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '409353a112304a9792ccf810ba286070'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:53.894239 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1213/2400] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:38:53 2022] DELETE /identity/v3/users/409353a112304a9792ccf810ba286070 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:38:54.201342 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104e754-2479-4b45-8a40-5d69334aeaef admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:54.202895 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-2104e754-2479-4b45-8a40-5d69334aeaef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:54.223350 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104e754-2479-4b45-8a40-5d69334aeaef admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:54.340511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982825ad-4e41-450d-a229-e3227e32e66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:54.342175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982825ad-4e41-450d-a229-e3227e32e66f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:54.342760 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982825ad-4e41-450d-a229-e3227e32e66f admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:54.343134 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982825ad-4e41-450d-a229-e3227e32e66f admin admin] PATH_INFO: `/v3/projects/aeb05fa6f70c406eba3e5bff61fb334d` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:54.347197 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982825ad-4e41-450d-a229-e3227e32e66f admin admin] RBAC: Authorizing `identity:delete_project(project_id=aeb05fa6f70c406eba3e5bff61fb334d)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:54.349202 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982825ad-4e41-450d-a229-e3227e32e66f admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:54.422324 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-982825ad-4e41-450d-a229-e3227e32e66f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aeb05fa6f70c406eba3e5bff61fb334d', 'request_id': 'req-982825ad-4e41-450d-a229-e3227e32e66f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:54.437048 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-982825ad-4e41-450d-a229-e3227e32e66f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aeb05fa6f70c406eba3e5bff61fb334d', 'request_id': 'req-982825ad-4e41-450d-a229-e3227e32e66f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:54.445061 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-982825ad-4e41-450d-a229-e3227e32e66f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aeb05fa6f70c406eba3e5bff61fb334d', 'request_id': 'req-982825ad-4e41-450d-a229-e3227e32e66f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:54.471159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-982825ad-4e41-450d-a229-e3227e32e66f admin admin] The token cache is being invalidate because project aeb05fa6f70c406eba3e5bff61fb334d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:38:54.471465 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-982825ad-4e41-450d-a229-e3227e32e66f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:54.472142 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-982825ad-4e41-450d-a229-e3227e32e66f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:38:54.473907 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1188/2401] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:38:54 2022] DELETE /identity/v3/projects/aeb05fa6f70c406eba3e5bff61fb334d => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:38:54.479290 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118a5e8-171c-419f-ab5e-de7366811462 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:54.479579 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118a5e8-171c-419f-ab5e-de7366811462 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:54.479857 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118a5e8-171c-419f-ab5e-de7366811462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:54.504462 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-2118a5e8-171c-419f-ab5e-de7366811462 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:38:54.505842 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-2118a5e8-171c-419f-ab5e-de7366811462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:54.576145 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:54.594077 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1214/2402] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:38:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:38:54.603271 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20489fa5-fec8-4caa-8bd1-4912a48dad01 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:54.603562 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20489fa5-fec8-4caa-8bd1-4912a48dad01 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:54.603848 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20489fa5-fec8-4caa-8bd1-4912a48dad01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:54.649254 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-20489fa5-fec8-4caa-8bd1-4912a48dad01 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:38:54.650586 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-20489fa5-fec8-4caa-8bd1-4912a48dad01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:54.675515 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1189/2403] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:38:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:38:54.676692 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118a5e8-171c-419f-ab5e-de7366811462 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:54.693646 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118a5e8-171c-419f-ab5e-de7366811462 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:54.703059 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a8b5b-46ca-4e1a-a66f-32fcd4698915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:54.703679 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a8b5b-46ca-4e1a-a66f-32fcd4698915 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:54.703957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a8b5b-46ca-4e1a-a66f-32fcd4698915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:54.704230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a8b5b-46ca-4e1a-a66f-32fcd4698915 admin admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:54.704886 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a8b5b-46ca-4e1a-a66f-32fcd4698915 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:38:54.705194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a8b5b-46ca-4e1a-a66f-32fcd4698915 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:54.707682 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a8b5b-46ca-4e1a-a66f-32fcd4698915 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:54.730453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1215/2404] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:38:54.733126 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20489fa5-fec8-4caa-8bd1-4912a48dad01 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:54.752331 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20489fa5-fec8-4caa-8bd1-4912a48dad01 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:54.759612 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3609573-964f-466b-9ce4-0c0b209a7505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:54.760244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3609573-964f-466b-9ce4-0c0b209a7505 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:54.760524 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3609573-964f-466b-9ce4-0c0b209a7505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:54.760795 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3609573-964f-466b-9ce4-0c0b209a7505 admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:54.761629 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3609573-964f-466b-9ce4-0c0b209a7505 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:54.763719 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3609573-964f-466b-9ce4-0c0b209a7505 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:54.811906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1190/2405] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:38:54 2022] POST /identity/v3/projects => generated 381 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:38:54.813248 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a8b5b-46ca-4e1a-a66f-32fcd4698915 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:54.838672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a8b5b-46ca-4e1a-a66f-32fcd4698915 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:54.920984 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ad2e6-6659-4325-a012-d89b604f27ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:54.921611 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ad2e6-6659-4325-a012-d89b604f27ef admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:54.921886 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ad2e6-6659-4325-a012-d89b604f27ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:54.922195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ad2e6-6659-4325-a012-d89b604f27ef admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:54.922857 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ad2e6-6659-4325-a012-d89b604f27ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:54.924780 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ad2e6-6659-4325-a012-d89b604f27ef admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:54.951717 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1216/2406] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:38:54 2022] POST /identity/v3/users => generated 291 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:38:54.953018 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3609573-964f-466b-9ce4-0c0b209a7505 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:54.965854 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3609573-964f-466b-9ce4-0c0b209a7505 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:54.973036 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ab994-6bf3-4b82-aeb1-df80aac7cb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:54.973665 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ab994-6bf3-4b82-aeb1-df80aac7cb55 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:54.973945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ab994-6bf3-4b82-aeb1-df80aac7cb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:54.974255 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ab994-6bf3-4b82-aeb1-df80aac7cb55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:54.974926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ab994-6bf3-4b82-aeb1-df80aac7cb55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:54.976630 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ab994-6bf3-4b82-aeb1-df80aac7cb55 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:54.997270 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1191/2407] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:38:55.002620 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ad2e6-6659-4325-a012-d89b604f27ef admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:55.019239 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ad2e6-6659-4325-a012-d89b604f27ef admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:55.035665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93afb6db-bf93-4ed3-9f8a-ee46e137afe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:55.036389 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afb6db-bf93-4ed3-9f8a-ee46e137afe5 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:55.036919 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afb6db-bf93-4ed3-9f8a-ee46e137afe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:55.037194 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afb6db-bf93-4ed3-9f8a-ee46e137afe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:55.037990 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93afb6db-bf93-4ed3-9f8a-ee46e137afe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:55.039753 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93afb6db-bf93-4ed3-9f8a-ee46e137afe5 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:55.058064 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1217/2408] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:38:55.059804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ab994-6bf3-4b82-aeb1-df80aac7cb55 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:55.072913 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ab994-6bf3-4b82-aeb1-df80aac7cb55 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:55.079943 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02baf4f-875d-405a-8558-e33c13711d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:55.080983 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02baf4f-875d-405a-8558-e33c13711d6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:55.081416 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02baf4f-875d-405a-8558-e33c13711d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:55.083275 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02baf4f-875d-405a-8558-e33c13711d6d admin admin] PATH_INFO: `/v3/projects/3e71ac778bf24f779d6d4c8c5de7e528/users/57fa5ee68e6f48e982c312e6bb3a6ce6/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:55.103765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:55.125215 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02baf4f-875d-405a-8558-e33c13711d6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e71ac778bf24f779d6d4c8c5de7e528, user_id=57fa5ee68e6f48e982c312e6bb3a6ce6, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:55.135098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02baf4f-875d-405a-8558-e33c13711d6d admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:55.158069 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1192/2409] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:38:55 2022] PUT /identity/v3/projects/3e71ac778bf24f779d6d4c8c5de7e528/users/57fa5ee68e6f48e982c312e6bb3a6ce6/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:38:55.165541 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a23ecbd-73b1-40b0-96bf-4299d4af6f4d None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:55.166137 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a23ecbd-73b1-40b0-96bf-4299d4af6f4d None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:55.166892 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a23ecbd-73b1-40b0-96bf-4299d4af6f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:55.196000 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-4a23ecbd-73b1-40b0-96bf-4299d4af6f4d None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:38:55.197522 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-4a23ecbd-73b1-40b0-96bf-4299d4af6f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:55.274103 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1218/2410] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:38:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:38:55.283272 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02baf4f-875d-405a-8558-e33c13711d6d admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:55.285876 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-a02baf4f-875d-405a-8558-e33c13711d6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:55.379819 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02baf4f-875d-405a-8558-e33c13711d6d admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:55.389459 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b6aca-00b2-4a14-b0b1-8fceff9460b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:55.390947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b6aca-00b2-4a14-b0b1-8fceff9460b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:55.391191 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b6aca-00b2-4a14-b0b1-8fceff9460b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:55.391797 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b6aca-00b2-4a14-b0b1-8fceff9460b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:55.419540 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b6aca-00b2-4a14-b0b1-8fceff9460b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:55.421134 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b6aca-00b2-4a14-b0b1-8fceff9460b9 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:55.437938 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1193/2411] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:38:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:38:56.335864 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0476dd-e478-4ffd-abfe-3924b9a3315f None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:56.336415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0476dd-e478-4ffd-abfe-3924b9a3315f None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:56.336524 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0476dd-e478-4ffd-abfe-3924b9a3315f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:56.371697 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-6a0476dd-e478-4ffd-abfe-3924b9a3315f None None] MFA Rules not processed for user `57fa5ee68e6f48e982c312e6bb3a6ce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:38:56.373848 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-6a0476dd-e478-4ffd-abfe-3924b9a3315f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:56.462369 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:38:56.491645 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b6aca-00b2-4a14-b0b1-8fceff9460b9 service neutron] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:56.493103 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-da5b6aca-00b2-4a14-b0b1-8fceff9460b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:56.500491 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1219/2412] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:38:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:38:56.608088 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b6aca-00b2-4a14-b0b1-8fceff9460b9 service neutron] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:56.613946 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e309d6-94ba-44e8-b2a5-5d5e7b8fdfe1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:56.614853 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e309d6-94ba-44e8-b2a5-5d5e7b8fdfe1 service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:56.615293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e309d6-94ba-44e8-b2a5-5d5e7b8fdfe1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:56.615722 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e309d6-94ba-44e8-b2a5-5d5e7b8fdfe1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:56.627868 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e309d6-94ba-44e8-b2a5-5d5e7b8fdfe1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:56.629641 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e309d6-94ba-44e8-b2a5-5d5e7b8fdfe1 service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:56.641835 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1194/2413] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:38:56 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:38:58.102108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede8798a-a210-4ee2-8673-c3301b05c470 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:58.102700 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede8798a-a210-4ee2-8673-c3301b05c470 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:58.102879 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede8798a-a210-4ee2-8673-c3301b05c470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:58.127575 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-ede8798a-a210-4ee2-8673-c3301b05c470 None None] MFA Rules not processed for user `57fa5ee68e6f48e982c312e6bb3a6ce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:38:58.128965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-ede8798a-a210-4ee2-8673-c3301b05c470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:58.139091 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1220/2414] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:38:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:38:58.480837 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e309d6-94ba-44e8-b2a5-5d5e7b8fdfe1 service cinder] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:58.482345 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-a8e309d6-94ba-44e8-b2a5-5d5e7b8fdfe1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:58.497979 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e309d6-94ba-44e8-b2a5-5d5e7b8fdfe1 service cinder] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:58.510401 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a3adf1-3e8e-46b4-9695-c6779323ea97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:58.511378 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3adf1-3e8e-46b4-9695-c6779323ea97 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:58.511663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3adf1-3e8e-46b4-9695-c6779323ea97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:58.511940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3adf1-3e8e-46b4-9695-c6779323ea97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:58.515038 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-b0a3adf1-3e8e-46b4-9695-c6779323ea97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:58.526942 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a3adf1-3e8e-46b4-9695-c6779323ea97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:58.528448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a3adf1-3e8e-46b4-9695-c6779323ea97 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:58.538213 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1195/2415] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:38:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:38:59.584942 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede8798a-a210-4ee2-8673-c3301b05c470 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:38:59.606788 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede8798a-a210-4ee2-8673-c3301b05c470 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:38:59.618133 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99064545-d8a5-4176-a4c5-36335cb8b0be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:38:59.618668 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99064545-d8a5-4176-a4c5-36335cb8b0be service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:38:59.618967 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99064545-d8a5-4176-a4c5-36335cb8b0be service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:38:59.619237 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99064545-d8a5-4176-a4c5-36335cb8b0be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:38:59.623679 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-99064545-d8a5-4176-a4c5-36335cb8b0be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:38:59.640504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99064545-d8a5-4176-a4c5-36335cb8b0be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:38:59.642590 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99064545-d8a5-4176-a4c5-36335cb8b0be service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:38:59.655051 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1221/2416] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:38:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:39:04.989749 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a3adf1-3e8e-46b4-9695-c6779323ea97 service neutron] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:04.992882 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-b0a3adf1-3e8e-46b4-9695-c6779323ea97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:05.016350 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a3adf1-3e8e-46b4-9695-c6779323ea97 service neutron] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:05.031439 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd15451-220e-4daf-81f3-bf80fd77dbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:05.032088 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd15451-220e-4daf-81f3-bf80fd77dbba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:05.032365 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd15451-220e-4daf-81f3-bf80fd77dbba admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:05.032638 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd15451-220e-4daf-81f3-bf80fd77dbba admin admin] PATH_INFO: `/v3/users/cdcfab461e534465a906c0eef201e423` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:05.034680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd15451-220e-4daf-81f3-bf80fd77dbba admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdcfab461e534465a906c0eef201e423)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:05.037618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd15451-220e-4daf-81f3-bf80fd77dbba admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:05.109737 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-5bd15451-220e-4daf-81f3-bf80fd77dbba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdcfab461e534465a906c0eef201e423'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:05.110543 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-5bd15451-220e-4daf-81f3-bf80fd77dbba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdcfab461e534465a906c0eef201e423'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:05.111243 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-5bd15451-220e-4daf-81f3-bf80fd77dbba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdcfab461e534465a906c0eef201e423'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:05.126943 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-5bd15451-220e-4daf-81f3-bf80fd77dbba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdcfab461e534465a906c0eef201e423'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:05.143307 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-5bd15451-220e-4daf-81f3-bf80fd77dbba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdcfab461e534465a906c0eef201e423'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:05.146079 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:05.168530 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1196/2417] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:39:04 2022] DELETE /identity/v3/users/cdcfab461e534465a906c0eef201e423 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:05.375864 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99064545-d8a5-4176-a4c5-36335cb8b0be service neutron] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:05.393593 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99064545-d8a5-4176-a4c5-36335cb8b0be service neutron] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:05.472027 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ec0b4-08db-4995-8c90-d18bc804a50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:05.472670 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ec0b4-08db-4995-8c90-d18bc804a50a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:05.472961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ec0b4-08db-4995-8c90-d18bc804a50a admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:05.473236 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ec0b4-08db-4995-8c90-d18bc804a50a admin admin] PATH_INFO: `/v3/projects/3c219db8370d4ef49f235020868144d7` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:05.474392 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ec0b4-08db-4995-8c90-d18bc804a50a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c219db8370d4ef49f235020868144d7)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:05.476842 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ec0b4-08db-4995-8c90-d18bc804a50a admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:05.528902 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-a44ec0b4-08db-4995-8c90-d18bc804a50a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c219db8370d4ef49f235020868144d7', 'request_id': 'req-a44ec0b4-08db-4995-8c90-d18bc804a50a'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:05.535184 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-a44ec0b4-08db-4995-8c90-d18bc804a50a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c219db8370d4ef49f235020868144d7', 'request_id': 'req-a44ec0b4-08db-4995-8c90-d18bc804a50a'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:05.543216 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-a44ec0b4-08db-4995-8c90-d18bc804a50a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c219db8370d4ef49f235020868144d7', 'request_id': 'req-a44ec0b4-08db-4995-8c90-d18bc804a50a'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:05.556734 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-a44ec0b4-08db-4995-8c90-d18bc804a50a admin admin] The token cache is being invalidate because project 3c219db8370d4ef49f235020868144d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:39:05.556734 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-a44ec0b4-08db-4995-8c90-d18bc804a50a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:05.556734 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-a44ec0b4-08db-4995-8c90-d18bc804a50a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:05.556734 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1222/2418] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:39:05 2022] DELETE /identity/v3/projects/3c219db8370d4ef49f235020868144d7 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:05.561153 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdedbc-69d8-4faa-b213-ad912efeca7f None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:05.561537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdedbc-69d8-4faa-b213-ad912efeca7f None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:05.561873 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdedbc-69d8-4faa-b213-ad912efeca7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:05.582113 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-3afdedbc-69d8-4faa-b213-ad912efeca7f None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:05.583364 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-3afdedbc-69d8-4faa-b213-ad912efeca7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:05.654260 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1197/2419] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:05.662331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831a80e-cc93-439e-933e-97cc9a77db8f None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:05.662617 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831a80e-cc93-439e-933e-97cc9a77db8f None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:05.662938 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831a80e-cc93-439e-933e-97cc9a77db8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:05.683327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-a831a80e-cc93-439e-933e-97cc9a77db8f None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:05.684621 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-a831a80e-cc93-439e-933e-97cc9a77db8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:05.696360 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1223/2420] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:05.697260 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afdedbc-69d8-4faa-b213-ad912efeca7f None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:05.707444 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afdedbc-69d8-4faa-b213-ad912efeca7f None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:05.713741 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4512ab2-0943-4d40-ad75-fb7de14b538a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:05.714497 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4512ab2-0943-4d40-ad75-fb7de14b538a admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:05.714874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4512ab2-0943-4d40-ad75-fb7de14b538a admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:05.715662 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4512ab2-0943-4d40-ad75-fb7de14b538a admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:05.716217 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4512ab2-0943-4d40-ad75-fb7de14b538a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:39:05.716696 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4512ab2-0943-4d40-ad75-fb7de14b538a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:05.718604 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4512ab2-0943-4d40-ad75-fb7de14b538a admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:05.736684 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a831a80e-cc93-439e-933e-97cc9a77db8f None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:05.738256 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1198/2421] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:39:05.746573 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a831a80e-cc93-439e-933e-97cc9a77db8f None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:05.751487 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7de391-860e-4bfc-a666-da8c0ab0b374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:05.752091 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7de391-860e-4bfc-a666-da8c0ab0b374 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:05.752365 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7de391-860e-4bfc-a666-da8c0ab0b374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:05.752830 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7de391-860e-4bfc-a666-da8c0ab0b374 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:05.753461 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7de391-860e-4bfc-a666-da8c0ab0b374 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:05.755375 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7de391-860e-4bfc-a666-da8c0ab0b374 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:05.792737 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1224/2422] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:39:05 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:39:05.804125 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4512ab2-0943-4d40-ad75-fb7de14b538a admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:05.817246 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4512ab2-0943-4d40-ad75-fb7de14b538a admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:05.888296 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb3a40-8ea0-4e16-9c1a-67583427f96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:05.888915 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb3a40-8ea0-4e16-9c1a-67583427f96b admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:05.889197 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb3a40-8ea0-4e16-9c1a-67583427f96b admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:05.889473 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb3a40-8ea0-4e16-9c1a-67583427f96b admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:05.890140 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb3a40-8ea0-4e16-9c1a-67583427f96b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:05.892115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb3a40-8ea0-4e16-9c1a-67583427f96b admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:05.922517 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1199/2423] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:39:05 2022] POST /identity/v3/users => generated 287 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:39:05.927014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7de391-860e-4bfc-a666-da8c0ab0b374 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:05.942798 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7de391-860e-4bfc-a666-da8c0ab0b374 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:05.948977 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341e625-f64e-4607-b346-e65e56680a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:05.949683 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341e625-f64e-4607-b346-e65e56680a52 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:05.949963 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341e625-f64e-4607-b346-e65e56680a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:05.950304 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341e625-f64e-4607-b346-e65e56680a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:05.950952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a341e625-f64e-4607-b346-e65e56680a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:05.952631 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a341e625-f64e-4607-b346-e65e56680a52 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:05.964634 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1225/2424] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:39:05.968786 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb3a40-8ea0-4e16-9c1a-67583427f96b admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:05.979897 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb3a40-8ea0-4e16-9c1a-67583427f96b admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:05.986395 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1968db9-0d4e-4bed-8f9f-e7e1f75bce83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:05.987642 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1968db9-0d4e-4bed-8f9f-e7e1f75bce83 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:05.988152 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1968db9-0d4e-4bed-8f9f-e7e1f75bce83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:05.988485 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1968db9-0d4e-4bed-8f9f-e7e1f75bce83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:05.989124 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1968db9-0d4e-4bed-8f9f-e7e1f75bce83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:05.990754 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1968db9-0d4e-4bed-8f9f-e7e1f75bce83 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:06.001766 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1200/2425] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:39:06.006161 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341e625-f64e-4607-b346-e65e56680a52 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:06.016893 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341e625-f64e-4607-b346-e65e56680a52 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:06.021663 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1a87b-6644-4c57-bde5-5ad4fe420ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:06.022929 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1a87b-6644-4c57-bde5-5ad4fe420ce2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:06.023230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1a87b-6644-4c57-bde5-5ad4fe420ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:06.023570 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1a87b-6644-4c57-bde5-5ad4fe420ce2 admin admin] PATH_INFO: `/v3/projects/fc3f6f7d9cca43ee8c7d6c4d936f47f2/users/867cd8ad1b854c7ab2665ce605310bf9/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:06.038286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f1a87b-6644-4c57-bde5-5ad4fe420ce2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc3f6f7d9cca43ee8c7d6c4d936f47f2, user_id=867cd8ad1b854c7ab2665ce605310bf9, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:06.041087 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f1a87b-6644-4c57-bde5-5ad4fe420ce2 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:06.050074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:06.055199 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1226/2426] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:39:06 2022] PUT /identity/v3/projects/fc3f6f7d9cca43ee8c7d6c4d936f47f2/users/867cd8ad1b854c7ab2665ce605310bf9/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:06.067094 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74209f4d-217b-4fbf-86e5-1e912f5cd204 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:06.067383 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74209f4d-217b-4fbf-86e5-1e912f5cd204 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:06.067659 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74209f4d-217b-4fbf-86e5-1e912f5cd204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:06.097439 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-74209f4d-217b-4fbf-86e5-1e912f5cd204 None None] MFA Rules not processed for user `867cd8ad1b854c7ab2665ce605310bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:06.098705 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-74209f4d-217b-4fbf-86e5-1e912f5cd204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:06.146493 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:06.168239 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1201/2427] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:06.172501 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1a87b-6644-4c57-bde5-5ad4fe420ce2 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:06.239175 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1a87b-6644-4c57-bde5-5ad4fe420ce2 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:06.243974 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b90034-c232-43f7-b298-39490e503c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:06.244583 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b90034-c232-43f7-b298-39490e503c52 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:06.244860 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b90034-c232-43f7-b298-39490e503c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:06.245130 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b90034-c232-43f7-b298-39490e503c52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:06.245758 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b90034-c232-43f7-b298-39490e503c52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:06.247736 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b90034-c232-43f7-b298-39490e503c52 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:06.275405 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1227/2428] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:39:06 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:39:06.279527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74209f4d-217b-4fbf-86e5-1e912f5cd204 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:06.290897 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74209f4d-217b-4fbf-86e5-1e912f5cd204 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:06.352690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84693ec-179d-4cba-8d00-2292dd66441d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:06.353299 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84693ec-179d-4cba-8d00-2292dd66441d admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:06.353583 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84693ec-179d-4cba-8d00-2292dd66441d admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:06.353850 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84693ec-179d-4cba-8d00-2292dd66441d admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:06.354520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84693ec-179d-4cba-8d00-2292dd66441d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:06.356698 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84693ec-179d-4cba-8d00-2292dd66441d admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:06.378719 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1202/2429] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:39:06 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:39:06.383485 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b90034-c232-43f7-b298-39490e503c52 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:06.393818 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b90034-c232-43f7-b298-39490e503c52 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:06.398472 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66259cf3-7214-4566-90b7-f158e211b53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:06.399057 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66259cf3-7214-4566-90b7-f158e211b53e admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:06.399336 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66259cf3-7214-4566-90b7-f158e211b53e admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:06.399608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66259cf3-7214-4566-90b7-f158e211b53e admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:06.400239 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66259cf3-7214-4566-90b7-f158e211b53e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:06.401827 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66259cf3-7214-4566-90b7-f158e211b53e admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:06.412393 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1228/2430] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:39:06.416734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84693ec-179d-4cba-8d00-2292dd66441d admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:06.428363 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84693ec-179d-4cba-8d00-2292dd66441d admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:06.433656 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce49e77-9a6b-4ca5-b228-04c66b406a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:06.434360 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce49e77-9a6b-4ca5-b228-04c66b406a4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:06.434640 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce49e77-9a6b-4ca5-b228-04c66b406a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:06.435027 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce49e77-9a6b-4ca5-b228-04c66b406a4e admin admin] PATH_INFO: `/v3/projects/e2bbb3953d57402389af343bacd37a5d/users/6f94173e296b4706ac40900e90a53de5/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:06.451392 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce49e77-9a6b-4ca5-b228-04c66b406a4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2bbb3953d57402389af343bacd37a5d, user_id=6f94173e296b4706ac40900e90a53de5, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:06.454110 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce49e77-9a6b-4ca5-b228-04c66b406a4e admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:06.469906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1203/2431] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:39:06 2022] PUT /identity/v3/projects/e2bbb3953d57402389af343bacd37a5d/users/6f94173e296b4706ac40900e90a53de5/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:06.473683 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66259cf3-7214-4566-90b7-f158e211b53e admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:06.487296 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66259cf3-7214-4566-90b7-f158e211b53e admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:06.551663 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eefc5a-1e61-47d4-8761-584a6aa19694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:06.552306 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eefc5a-1e61-47d4-8761-584a6aa19694 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:06.552889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eefc5a-1e61-47d4-8761-584a6aa19694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:06.553167 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eefc5a-1e61-47d4-8761-584a6aa19694 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:06.554423 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eefc5a-1e61-47d4-8761-584a6aa19694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:06.556183 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eefc5a-1e61-47d4-8761-584a6aa19694 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:06.567467 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1229/2432] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:39:06.572530 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce49e77-9a6b-4ca5-b228-04c66b406a4e admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:06.583278 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce49e77-9a6b-4ca5-b228-04c66b406a4e admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:06.588163 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250a9d3-c07f-46ad-8222-3bad28918e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:06.588838 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250a9d3-c07f-46ad-8222-3bad28918e21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:06.589246 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250a9d3-c07f-46ad-8222-3bad28918e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:06.589546 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250a9d3-c07f-46ad-8222-3bad28918e21 admin admin] PATH_INFO: `/v3/projects/e2bbb3953d57402389af343bacd37a5d/users/6f94173e296b4706ac40900e90a53de5/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:06.591552 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250a9d3-c07f-46ad-8222-3bad28918e21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2bbb3953d57402389af343bacd37a5d, user_id=6f94173e296b4706ac40900e90a53de5, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:06.594296 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250a9d3-c07f-46ad-8222-3bad28918e21 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:06.606590 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1204/2433] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:39:06 2022] PUT /identity/v3/projects/e2bbb3953d57402389af343bacd37a5d/users/6f94173e296b4706ac40900e90a53de5/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:39:06.620337 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cbefa-4597-414e-a954-5aa89608ec74 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:06.620690 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cbefa-4597-414e-a954-5aa89608ec74 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:06.621069 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cbefa-4597-414e-a954-5aa89608ec74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:06.650009 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-8d9cbefa-4597-414e-a954-5aa89608ec74 None None] MFA Rules not processed for user `6f94173e296b4706ac40900e90a53de5`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:06.651253 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-8d9cbefa-4597-414e-a954-5aa89608ec74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:06.711165 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:06.721463 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1230/2434] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:06 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:06.733509 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250a9d3-c07f-46ad-8222-3bad28918e21 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:06.735388 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-3250a9d3-c07f-46ad-8222-3bad28918e21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:06.795510 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250a9d3-c07f-46ad-8222-3bad28918e21 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:06.799222 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def54d8-132e-4e2f-8de3-7245c505b44c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:06.799829 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def54d8-132e-4e2f-8de3-7245c505b44c service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:06.800105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def54d8-132e-4e2f-8de3-7245c505b44c service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:06.800373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def54d8-132e-4e2f-8de3-7245c505b44c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:06.810232 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def54d8-132e-4e2f-8de3-7245c505b44c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:06.811661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def54d8-132e-4e2f-8de3-7245c505b44c service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:06.821152 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1205/2435] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:39:06 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:39:07.227363 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cbefa-4597-414e-a954-5aa89608ec74 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:07.292304 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cbefa-4597-414e-a954-5aa89608ec74 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:07.296827 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9a366-6001-4694-986d-97c99218fb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:07.297448 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9a366-6001-4694-986d-97c99218fb4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:07.297722 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9a366-6001-4694-986d-97c99218fb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:07.297992 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9a366-6001-4694-986d-97c99218fb4c admin admin] PATH_INFO: `/v3/users/867cd8ad1b854c7ab2665ce605310bf9` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:07.299152 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d9a366-6001-4694-986d-97c99218fb4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=867cd8ad1b854c7ab2665ce605310bf9)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:07.301171 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d9a366-6001-4694-986d-97c99218fb4c admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:07.350870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-a2d9a366-6001-4694-986d-97c99218fb4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '867cd8ad1b854c7ab2665ce605310bf9'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:07.351619 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-a2d9a366-6001-4694-986d-97c99218fb4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '867cd8ad1b854c7ab2665ce605310bf9'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:07.359901 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-a2d9a366-6001-4694-986d-97c99218fb4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '867cd8ad1b854c7ab2665ce605310bf9'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:07.372828 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-a2d9a366-6001-4694-986d-97c99218fb4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '867cd8ad1b854c7ab2665ce605310bf9'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:07.373495 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-a2d9a366-6001-4694-986d-97c99218fb4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '867cd8ad1b854c7ab2665ce605310bf9'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:07.387132 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1231/2436] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:39:07 2022] DELETE /identity/v3/users/867cd8ad1b854c7ab2665ce605310bf9 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:07.391604 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c497770-42c4-4453-a0af-d6777de8b3cc None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:07.392030 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c497770-42c4-4453-a0af-d6777de8b3cc None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:07.392192 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c497770-42c4-4453-a0af-d6777de8b3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:07.410866 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-2c497770-42c4-4453-a0af-d6777de8b3cc None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:07.412068 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-2c497770-42c4-4453-a0af-d6777de8b3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:07.452464 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:07.474361 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1206/2437] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:07.484945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9a366-6001-4694-986d-97c99218fb4c admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:07.486624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-a2d9a366-6001-4694-986d-97c99218fb4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:07.545994 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9a366-6001-4694-986d-97c99218fb4c admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:07.549575 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8d28b-46e1-4e4f-87ad-edfd6e540714 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:07.550213 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8d28b-46e1-4e4f-87ad-edfd6e540714 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:07.550486 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8d28b-46e1-4e4f-87ad-edfd6e540714 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:07.550757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8d28b-46e1-4e4f-87ad-edfd6e540714 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:07.560573 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc8d28b-46e1-4e4f-87ad-edfd6e540714 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:07.562305 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc8d28b-46e1-4e4f-87ad-edfd6e540714 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:07.572497 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1232/2438] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:39:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:39:07.891688 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c497770-42c4-4453-a0af-d6777de8b3cc None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:07.893193 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-2c497770-42c4-4453-a0af-d6777de8b3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:07.909152 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c497770-42c4-4453-a0af-d6777de8b3cc None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:07.915492 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a0d24-e72f-4784-8d72-585156ac7ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:07.916124 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a0d24-e72f-4784-8d72-585156ac7ad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:07.916397 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a0d24-e72f-4784-8d72-585156ac7ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:07.916663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a0d24-e72f-4784-8d72-585156ac7ad1 admin admin] PATH_INFO: `/v3/projects/fc3f6f7d9cca43ee8c7d6c4d936f47f2` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:07.917886 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391a0d24-e72f-4784-8d72-585156ac7ad1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc3f6f7d9cca43ee8c7d6c4d936f47f2)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:07.919971 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391a0d24-e72f-4784-8d72-585156ac7ad1 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:07.954393 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85582700-b37b-40ba-86d3-07245303fa6b None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:07.954768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85582700-b37b-40ba-86d3-07245303fa6b None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:07.955308 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85582700-b37b-40ba-86d3-07245303fa6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:07.978841 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-391a0d24-e72f-4784-8d72-585156ac7ad1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc3f6f7d9cca43ee8c7d6c4d936f47f2', 'request_id': 'req-391a0d24-e72f-4784-8d72-585156ac7ad1'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:07.979354 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-85582700-b37b-40ba-86d3-07245303fa6b None None] MFA Rules not processed for user `9158557f1a2947e2815dc8d586d8b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:07.980990 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-85582700-b37b-40ba-86d3-07245303fa6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:07.986303 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-391a0d24-e72f-4784-8d72-585156ac7ad1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc3f6f7d9cca43ee8c7d6c4d936f47f2', 'request_id': 'req-391a0d24-e72f-4784-8d72-585156ac7ad1'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:07.993555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-391a0d24-e72f-4784-8d72-585156ac7ad1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc3f6f7d9cca43ee8c7d6c4d936f47f2', 'request_id': 'req-391a0d24-e72f-4784-8d72-585156ac7ad1'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:07.997078 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:08.012267 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-391a0d24-e72f-4784-8d72-585156ac7ad1 admin admin] The token cache is being invalidate because project fc3f6f7d9cca43ee8c7d6c4d936f47f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:39:08.012563 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-391a0d24-e72f-4784-8d72-585156ac7ad1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:08.013859 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-391a0d24-e72f-4784-8d72-585156ac7ad1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:08.016968 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1207/2439] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:39:07 2022] DELETE /identity/v3/projects/fc3f6f7d9cca43ee8c7d6c4d936f47f2 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:08.021919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a0d24-e72f-4784-8d72-585156ac7ad1 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:08.024667 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-391a0d24-e72f-4784-8d72-585156ac7ad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:08.072696 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1233/2440] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Sep 26 10:39:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:08.089187 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85582700-b37b-40ba-86d3-07245303fa6b None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:08.090374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-85582700-b37b-40ba-86d3-07245303fa6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:08.103804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a0d24-e72f-4784-8d72-585156ac7ad1 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:08.109814 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a32cd-5e26-4420-9f2e-8390f90a786a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:08.110646 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a32cd-5e26-4420-9f2e-8390f90a786a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:08.111022 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a32cd-5e26-4420-9f2e-8390f90a786a admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:08.111357 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a32cd-5e26-4420-9f2e-8390f90a786a admin admin] PATH_INFO: `/v3/users/6f94173e296b4706ac40900e90a53de5` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:08.112736 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a32cd-5e26-4420-9f2e-8390f90a786a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f94173e296b4706ac40900e90a53de5)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:08.115496 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a32cd-5e26-4420-9f2e-8390f90a786a admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:08.171731 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-0f4a32cd-5e26-4420-9f2e-8390f90a786a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f94173e296b4706ac40900e90a53de5'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:08.172442 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-0f4a32cd-5e26-4420-9f2e-8390f90a786a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f94173e296b4706ac40900e90a53de5'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:08.173135 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-0f4a32cd-5e26-4420-9f2e-8390f90a786a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f94173e296b4706ac40900e90a53de5'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:08.180877 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-0f4a32cd-5e26-4420-9f2e-8390f90a786a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f94173e296b4706ac40900e90a53de5'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:08.184092 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85582700-b37b-40ba-86d3-07245303fa6b None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:08.189252 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:08.190100 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758071c-4dac-455b-9f75-8360f6c2994d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e6be34fc93a4551a83e5b8380188a6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:08.190718 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758071c-4dac-455b-9f75-8360f6c2994d service swift] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:08.191133 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758071c-4dac-455b-9f75-8360f6c2994d service swift] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:08.191475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758071c-4dac-455b-9f75-8360f6c2994d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:08.193326 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-5758071c-4dac-455b-9f75-8360f6c2994d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:08.197142 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-0f4a32cd-5e26-4420-9f2e-8390f90a786a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f94173e296b4706ac40900e90a53de5'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:08.206219 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5758071c-4dac-455b-9f75-8360f6c2994d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:08.207817 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5758071c-4dac-455b-9f75-8360f6c2994d service swift] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:08.209561 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1208/2441] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:39:08 2022] DELETE /identity/v3/users/6f94173e296b4706ac40900e90a53de5 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:08.217912 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1234/2442] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 10:39:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 10:39:08.664711 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a32cd-5e26-4420-9f2e-8390f90a786a admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:08.666287 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-0f4a32cd-5e26-4420-9f2e-8390f90a786a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:08.832059 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a32cd-5e26-4420-9f2e-8390f90a786a admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:08.837724 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99163710-5467-424f-baac-8b4ff3579b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:08.838412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99163710-5467-424f-baac-8b4ff3579b44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:08.838691 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99163710-5467-424f-baac-8b4ff3579b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:08.838964 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99163710-5467-424f-baac-8b4ff3579b44 admin admin] PATH_INFO: `/v3/projects/e2bbb3953d57402389af343bacd37a5d` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:08.840117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99163710-5467-424f-baac-8b4ff3579b44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2bbb3953d57402389af343bacd37a5d)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:08.842598 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99163710-5467-424f-baac-8b4ff3579b44 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:08.910081 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-99163710-5467-424f-baac-8b4ff3579b44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2bbb3953d57402389af343bacd37a5d', 'request_id': 'req-99163710-5467-424f-baac-8b4ff3579b44'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:08.919393 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-99163710-5467-424f-baac-8b4ff3579b44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2bbb3953d57402389af343bacd37a5d', 'request_id': 'req-99163710-5467-424f-baac-8b4ff3579b44'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:08.929582 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-99163710-5467-424f-baac-8b4ff3579b44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2bbb3953d57402389af343bacd37a5d', 'request_id': 'req-99163710-5467-424f-baac-8b4ff3579b44'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:08.957028 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-99163710-5467-424f-baac-8b4ff3579b44 admin admin] The token cache is being invalidate because project e2bbb3953d57402389af343bacd37a5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:39:08.958218 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-99163710-5467-424f-baac-8b4ff3579b44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:08.961072 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-99163710-5467-424f-baac-8b4ff3579b44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:08.967825 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1209/2443] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:39:08 2022] DELETE /identity/v3/projects/e2bbb3953d57402389af343bacd37a5d => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:08.972862 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa12d3b-6fc4-4baa-8f19-c911aa92c995 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:08.974714 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa12d3b-6fc4-4baa-8f19-c911aa92c995 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:08.976054 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa12d3b-6fc4-4baa-8f19-c911aa92c995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:09.022932 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-bfa12d3b-6fc4-4baa-8f19-c911aa92c995 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:09.025913 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-bfa12d3b-6fc4-4baa-8f19-c911aa92c995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:09.175627 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1235/2444] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:09.188631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75884480-2f11-499f-8677-a3f50d4f9dca None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:09.188912 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75884480-2f11-499f-8677-a3f50d4f9dca None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:09.189182 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75884480-2f11-499f-8677-a3f50d4f9dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:09.258155 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-75884480-2f11-499f-8677-a3f50d4f9dca None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:09.259540 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-75884480-2f11-499f-8677-a3f50d4f9dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:09.271864 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1210/2445] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:09.273319 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa12d3b-6fc4-4baa-8f19-c911aa92c995 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:09.299622 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa12d3b-6fc4-4baa-8f19-c911aa92c995 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:09.328421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b4875-dbdd-4a74-8d2d-39a477cd5871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:09.329421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b4875-dbdd-4a74-8d2d-39a477cd5871 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:09.329789 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b4875-dbdd-4a74-8d2d-39a477cd5871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:09.330091 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b4875-dbdd-4a74-8d2d-39a477cd5871 admin admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:09.331174 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b4875-dbdd-4a74-8d2d-39a477cd5871 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:39:09.331840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b4875-dbdd-4a74-8d2d-39a477cd5871 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:09.335176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b4875-dbdd-4a74-8d2d-39a477cd5871 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:09.370241 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1236/2446] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:39:09.374406 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75884480-2f11-499f-8677-a3f50d4f9dca None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:09.387610 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75884480-2f11-499f-8677-a3f50d4f9dca None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:09.403596 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c6340-0e8b-4e39-a758-a40480fcf353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:09.404224 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c6340-0e8b-4e39-a758-a40480fcf353 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:09.404532 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c6340-0e8b-4e39-a758-a40480fcf353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:09.404796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c6340-0e8b-4e39-a758-a40480fcf353 admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:09.405409 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02c6340-0e8b-4e39-a758-a40480fcf353 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:09.407360 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02c6340-0e8b-4e39-a758-a40480fcf353 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:09.465289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1211/2447] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:39:09 2022] POST /identity/v3/projects => generated 357 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:39:09.469108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b4875-dbdd-4a74-8d2d-39a477cd5871 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:09.488295 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b4875-dbdd-4a74-8d2d-39a477cd5871 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:09.554974 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:09.588310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca76835-9bd4-4710-95d3-10ecb646c2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:09.589390 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca76835-9bd4-4710-95d3-10ecb646c2c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:09.589727 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca76835-9bd4-4710-95d3-10ecb646c2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:09.590098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca76835-9bd4-4710-95d3-10ecb646c2c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:09.590887 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca76835-9bd4-4710-95d3-10ecb646c2c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:09.593061 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca76835-9bd4-4710-95d3-10ecb646c2c0 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:09.650060 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1237/2448] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:39:09 2022] POST /identity/v3/users => generated 279 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:39:09.650896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c6340-0e8b-4e39-a758-a40480fcf353 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:09.672582 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c6340-0e8b-4e39-a758-a40480fcf353 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:09.693843 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b3345-180f-4f31-b97b-0610183ca720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:09.694930 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b3345-180f-4f31-b97b-0610183ca720 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:09.695860 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b3345-180f-4f31-b97b-0610183ca720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:09.696401 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b3345-180f-4f31-b97b-0610183ca720 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:09.700045 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b3345-180f-4f31-b97b-0610183ca720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:09.702200 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b3345-180f-4f31-b97b-0610183ca720 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:09.728558 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1212/2449] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:39:09.733118 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca76835-9bd4-4710-95d3-10ecb646c2c0 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:09.765943 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca76835-9bd4-4710-95d3-10ecb646c2c0 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:09.782818 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda1424-5c07-4c6e-b658-ccb87a43e3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:09.783562 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda1424-5c07-4c6e-b658-ccb87a43e3c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:09.783851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda1424-5c07-4c6e-b658-ccb87a43e3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:09.784117 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda1424-5c07-4c6e-b658-ccb87a43e3c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:09.785084 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bda1424-5c07-4c6e-b658-ccb87a43e3c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:09.786890 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bda1424-5c07-4c6e-b658-ccb87a43e3c8 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:09.815734 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1238/2450] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:39:09.819944 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b3345-180f-4f31-b97b-0610183ca720 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:09.832670 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b3345-180f-4f31-b97b-0610183ca720 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:09.837593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8cc0a0-ba9e-4134-bab0-63dcd52078b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:09.838589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cc0a0-ba9e-4134-bab0-63dcd52078b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:09.838908 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cc0a0-ba9e-4134-bab0-63dcd52078b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:09.839268 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cc0a0-ba9e-4134-bab0-63dcd52078b9 admin admin] PATH_INFO: `/v3/projects/64d90d67489f4af8af66cf940fa2e84f/users/4b35b391a93b49bcab09b72d07f7e9e1/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:09.860534 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8cc0a0-ba9e-4134-bab0-63dcd52078b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64d90d67489f4af8af66cf940fa2e84f, user_id=4b35b391a93b49bcab09b72d07f7e9e1, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:09.864075 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8cc0a0-ba9e-4134-bab0-63dcd52078b9 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:09.897180 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1213/2451] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:39:09 2022] PUT /identity/v3/projects/64d90d67489f4af8af66cf940fa2e84f/users/4b35b391a93b49bcab09b72d07f7e9e1/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:39:09.899548 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac547e-bf96-48e7-a24f-592aa971f027 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:09.900112 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac547e-bf96-48e7-a24f-592aa971f027 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:09.900663 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac547e-bf96-48e7-a24f-592aa971f027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:09.936742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-f9ac547e-bf96-48e7-a24f-592aa971f027 None None] MFA Rules not processed for user `4b35b391a93b49bcab09b72d07f7e9e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:09.938489 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-f9ac547e-bf96-48e7-a24f-592aa971f027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:10.035986 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1239/2452] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:09 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:10.037424 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8cc0a0-ba9e-4134-bab0-63dcd52078b9 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:10.059123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:10.120397 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8cc0a0-ba9e-4134-bab0-63dcd52078b9 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:10.125575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8124b9b-a7cd-4e75-8921-7e425781dc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:10.126257 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8124b9b-a7cd-4e75-8921-7e425781dc87 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:10.126530 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8124b9b-a7cd-4e75-8921-7e425781dc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:10.126797 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8124b9b-a7cd-4e75-8921-7e425781dc87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:10.127429 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8124b9b-a7cd-4e75-8921-7e425781dc87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:10.129352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8124b9b-a7cd-4e75-8921-7e425781dc87 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:10.165278 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1214/2453] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:39:10 2022] POST /identity/v3/projects => generated 357 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:39:10.169656 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac547e-bf96-48e7-a24f-592aa971f027 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:10.196081 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac547e-bf96-48e7-a24f-592aa971f027 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:10.273421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe31481-9753-490d-a797-9e1d7d2227a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:10.274523 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe31481-9753-490d-a797-9e1d7d2227a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:10.275017 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe31481-9753-490d-a797-9e1d7d2227a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:10.275529 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe31481-9753-490d-a797-9e1d7d2227a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:10.276599 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe31481-9753-490d-a797-9e1d7d2227a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:10.281736 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe31481-9753-490d-a797-9e1d7d2227a3 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:10.322199 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1240/2454] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:39:10 2022] POST /identity/v3/users => generated 285 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:39:10.328380 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8124b9b-a7cd-4e75-8921-7e425781dc87 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:10.339249 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8124b9b-a7cd-4e75-8921-7e425781dc87 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:10.350466 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b41195c-ba36-476a-989d-b4e4b5db8f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:10.351607 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41195c-ba36-476a-989d-b4e4b5db8f70 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:10.352029 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41195c-ba36-476a-989d-b4e4b5db8f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:10.352309 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41195c-ba36-476a-989d-b4e4b5db8f70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:10.352969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b41195c-ba36-476a-989d-b4e4b5db8f70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:10.354657 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b41195c-ba36-476a-989d-b4e4b5db8f70 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:10.378084 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1215/2455] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:39:10.380965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe31481-9753-490d-a797-9e1d7d2227a3 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:10.392534 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe31481-9753-490d-a797-9e1d7d2227a3 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:10.401289 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f2e37-70a4-4a6f-b13f-631f80b77677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:10.401638 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f2e37-70a4-4a6f-b13f-631f80b77677 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:10.401919 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f2e37-70a4-4a6f-b13f-631f80b77677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:10.402224 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f2e37-70a4-4a6f-b13f-631f80b77677 admin admin] PATH_INFO: `/v3/projects/64fd4fa84fe645048c2fae36676e5dd0/users/d2ae77b932af41979998646f9e2841c7/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:10.429377 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952f2e37-70a4-4a6f-b13f-631f80b77677 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64fd4fa84fe645048c2fae36676e5dd0, user_id=d2ae77b932af41979998646f9e2841c7, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:10.432162 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952f2e37-70a4-4a6f-b13f-631f80b77677 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:10.453339 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1241/2456] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:39:10 2022] PUT /identity/v3/projects/64fd4fa84fe645048c2fae36676e5dd0/users/d2ae77b932af41979998646f9e2841c7/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:10.457326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b41195c-ba36-476a-989d-b4e4b5db8f70 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:10.469105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b41195c-ba36-476a-989d-b4e4b5db8f70 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:10.552567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a69475-c61b-412f-a8ce-858d0b8a7304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:10.553188 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a69475-c61b-412f-a8ce-858d0b8a7304 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:10.554186 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a69475-c61b-412f-a8ce-858d0b8a7304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:10.554465 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a69475-c61b-412f-a8ce-858d0b8a7304 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:10.555139 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a69475-c61b-412f-a8ce-858d0b8a7304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:10.557442 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a69475-c61b-412f-a8ce-858d0b8a7304 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:10.587243 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1216/2457] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:39:10.591648 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f2e37-70a4-4a6f-b13f-631f80b77677 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:10.606881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f2e37-70a4-4a6f-b13f-631f80b77677 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:10.613687 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ec4d4-5c61-43fc-b364-d4a0f3d969ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:10.614511 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ec4d4-5c61-43fc-b364-d4a0f3d969ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:10.621136 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ec4d4-5c61-43fc-b364-d4a0f3d969ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:10.621136 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ec4d4-5c61-43fc-b364-d4a0f3d969ec admin admin] PATH_INFO: `/v3/projects/64fd4fa84fe645048c2fae36676e5dd0/users/d2ae77b932af41979998646f9e2841c7/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:10.621136 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ec4d4-5c61-43fc-b364-d4a0f3d969ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=64fd4fa84fe645048c2fae36676e5dd0, user_id=d2ae77b932af41979998646f9e2841c7, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:10.621758 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ec4d4-5c61-43fc-b364-d4a0f3d969ec admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:10.628711 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:10.657910 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1242/2458] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:39:10 2022] PUT /identity/v3/projects/64fd4fa84fe645048c2fae36676e5dd0/users/d2ae77b932af41979998646f9e2841c7/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:10.669349 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67adab-fa25-4504-8707-dd793d816e3d None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:10.669711 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67adab-fa25-4504-8707-dd793d816e3d None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:10.669987 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67adab-fa25-4504-8707-dd793d816e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:10.733260 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-ce67adab-fa25-4504-8707-dd793d816e3d None None] MFA Rules not processed for user `d2ae77b932af41979998646f9e2841c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:10.734540 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-ce67adab-fa25-4504-8707-dd793d816e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:10.836154 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1217/2459] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:10 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:10.845570 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ec4d4-5c61-43fc-b364-d4a0f3d969ec admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:10.847538 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-1f7ec4d4-5c61-43fc-b364-d4a0f3d969ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:10.925321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ec4d4-5c61-43fc-b364-d4a0f3d969ec admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:10.931766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49c497-9298-4186-ad3b-647fb450737a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:10.932613 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49c497-9298-4186-ad3b-647fb450737a service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:10.933045 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49c497-9298-4186-ad3b-647fb450737a service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:10.933502 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49c497-9298-4186-ad3b-647fb450737a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:10.945057 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef49c497-9298-4186-ad3b-647fb450737a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:10.947084 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef49c497-9298-4186-ad3b-647fb450737a service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:10.960860 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1243/2460] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:39:10 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:39:11.081379 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67adab-fa25-4504-8707-dd793d816e3d None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:11.092335 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67adab-fa25-4504-8707-dd793d816e3d None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:11.096674 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17407da-1368-4b3d-bb72-ab85efb34a15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:11.097514 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17407da-1368-4b3d-bb72-ab85efb34a15 service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:11.097908 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17407da-1368-4b3d-bb72-ab85efb34a15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:11.098269 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17407da-1368-4b3d-bb72-ab85efb34a15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:11.109152 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:11.171854 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17407da-1368-4b3d-bb72-ab85efb34a15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:11.173873 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17407da-1368-4b3d-bb72-ab85efb34a15 service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:11.184010 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1218/2461] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:39:11 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:39:19.269319 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49c497-9298-4186-ad3b-647fb450737a service cinder] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:19.270789 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-ef49c497-9298-4186-ad3b-647fb450737a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:19.343711 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49c497-9298-4186-ad3b-647fb450737a service cinder] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:19.349161 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a812a4b-50fb-4489-b8ee-26f11d5200a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:19.349758 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a812a4b-50fb-4489-b8ee-26f11d5200a1 service nova] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:19.350063 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a812a4b-50fb-4489-b8ee-26f11d5200a1 service nova] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:19.350904 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a812a4b-50fb-4489-b8ee-26f11d5200a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:19.352596 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-7a812a4b-50fb-4489-b8ee-26f11d5200a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:19.410834 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a812a4b-50fb-4489-b8ee-26f11d5200a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:19.412493 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a812a4b-50fb-4489-b8ee-26f11d5200a1 service nova] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:19.422276 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1244/2462] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:39:19 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:39:21.179645 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=25 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:22.254558 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e0dd4-af6c-466e-afd0-e876a4f41ec2 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:22.255160 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e0dd4-af6c-466e-afd0-e876a4f41ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:22.255160 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e0dd4-af6c-466e-afd0-e876a4f41ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:22.276622 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-3b0e0dd4-af6c-466e-afd0-e876a4f41ec2 None None] MFA Rules not processed for user `9158557f1a2947e2815dc8d586d8b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:22.278525 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-3b0e0dd4-af6c-466e-afd0-e876a4f41ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:22.339365 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1219/2463] 10.222.0.49 () {58 vars in 967 bytes} [Mon Sep 26 10:39:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:22.363293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a812a4b-50fb-4489-b8ee-26f11d5200a1 service nova] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:22.364814 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-7a812a4b-50fb-4489-b8ee-26f11d5200a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:22.411221 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:22.443666 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a812a4b-50fb-4489-b8ee-26f11d5200a1 service nova] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:22.449401 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fbcf73-00af-47cf-a2f9-981f433713e6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e6be34fc93a4551a83e5b8380188a6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:22.450048 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbcf73-00af-47cf-a2f9-981f433713e6 service swift] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:22.450319 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbcf73-00af-47cf-a2f9-981f433713e6 service swift] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:22.450588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbcf73-00af-47cf-a2f9-981f433713e6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:22.458805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fbcf73-00af-47cf-a2f9-981f433713e6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:22.460923 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fbcf73-00af-47cf-a2f9-981f433713e6 service swift] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:22.468994 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1245/2464] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 10:39:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 10:39:26.488599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e0dd4-af6c-466e-afd0-e876a4f41ec2 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:26.554113 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e0dd4-af6c-466e-afd0-e876a4f41ec2 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:26.558718 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3055759-3a5e-4833-8afc-1359d001c353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:26.559349 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3055759-3a5e-4833-8afc-1359d001c353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:26.559748 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3055759-3a5e-4833-8afc-1359d001c353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:26.560025 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3055759-3a5e-4833-8afc-1359d001c353 admin admin] PATH_INFO: `/v3/users/4b35b391a93b49bcab09b72d07f7e9e1` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:26.561424 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3055759-3a5e-4833-8afc-1359d001c353 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b35b391a93b49bcab09b72d07f7e9e1)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:26.563569 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3055759-3a5e-4833-8afc-1359d001c353 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:26.611064 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-a3055759-3a5e-4833-8afc-1359d001c353 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b35b391a93b49bcab09b72d07f7e9e1'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:26.611721 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-a3055759-3a5e-4833-8afc-1359d001c353 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b35b391a93b49bcab09b72d07f7e9e1'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:26.612328 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-a3055759-3a5e-4833-8afc-1359d001c353 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b35b391a93b49bcab09b72d07f7e9e1'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:26.619288 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-a3055759-3a5e-4833-8afc-1359d001c353 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b35b391a93b49bcab09b72d07f7e9e1'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:26.631978 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-a3055759-3a5e-4833-8afc-1359d001c353 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b35b391a93b49bcab09b72d07f7e9e1'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:26.644443 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1220/2465] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:39:26 2022] DELETE /identity/v3/users/4b35b391a93b49bcab09b72d07f7e9e1 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:26.649773 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39359656-cdb3-4b27-be83-5d7f8c8e54eb None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:26.650341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39359656-cdb3-4b27-be83-5d7f8c8e54eb None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:26.650540 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39359656-cdb3-4b27-be83-5d7f8c8e54eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:26.670234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-39359656-cdb3-4b27-be83-5d7f8c8e54eb None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:26.671439 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-39359656-cdb3-4b27-be83-5d7f8c8e54eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:26.733536 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1246/2466] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:26.742982 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3055759-3a5e-4833-8afc-1359d001c353 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:26.744453 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-a3055759-3a5e-4833-8afc-1359d001c353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:26.778880 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:26.820685 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3055759-3a5e-4833-8afc-1359d001c353 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:26.824258 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd82a1-e1f5-45b3-8259-81c6c4c6b1c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:26.824874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd82a1-e1f5-45b3-8259-81c6c4c6b1c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:26.825154 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd82a1-e1f5-45b3-8259-81c6c4c6b1c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:26.825437 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd82a1-e1f5-45b3-8259-81c6c4c6b1c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:26.835538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfd82a1-e1f5-45b3-8259-81c6c4c6b1c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:26.837141 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfd82a1-e1f5-45b3-8259-81c6c4c6b1c1 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:26.847792 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1221/2467] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:39:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:39:27.129584 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39359656-cdb3-4b27-be83-5d7f8c8e54eb None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:27.131012 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-39359656-cdb3-4b27-be83-5d7f8c8e54eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:27.143586 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39359656-cdb3-4b27-be83-5d7f8c8e54eb None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:27.148994 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3913831-9196-4a75-8edc-7b1cde031040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:27.149720 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3913831-9196-4a75-8edc-7b1cde031040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:27.150062 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3913831-9196-4a75-8edc-7b1cde031040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:27.150394 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3913831-9196-4a75-8edc-7b1cde031040 admin admin] PATH_INFO: `/v3/projects/64d90d67489f4af8af66cf940fa2e84f` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:27.151666 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3913831-9196-4a75-8edc-7b1cde031040 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64d90d67489f4af8af66cf940fa2e84f)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:27.153935 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3913831-9196-4a75-8edc-7b1cde031040 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:27.204542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c3913831-9196-4a75-8edc-7b1cde031040 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64d90d67489f4af8af66cf940fa2e84f', 'request_id': 'req-c3913831-9196-4a75-8edc-7b1cde031040'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:27.211542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c3913831-9196-4a75-8edc-7b1cde031040 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64d90d67489f4af8af66cf940fa2e84f', 'request_id': 'req-c3913831-9196-4a75-8edc-7b1cde031040'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:27.221116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c3913831-9196-4a75-8edc-7b1cde031040 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64d90d67489f4af8af66cf940fa2e84f', 'request_id': 'req-c3913831-9196-4a75-8edc-7b1cde031040'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:27.232325 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c3913831-9196-4a75-8edc-7b1cde031040 admin admin] The token cache is being invalidate because project 64d90d67489f4af8af66cf940fa2e84f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:39:27.232612 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c3913831-9196-4a75-8edc-7b1cde031040 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:27.233264 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-c3913831-9196-4a75-8edc-7b1cde031040 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:27.234805 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1247/2468] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:39:27 2022] DELETE /identity/v3/projects/64d90d67489f4af8af66cf940fa2e84f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:27.238925 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd82a1-e1f5-45b3-8259-81c6c4c6b1c1 service neutron] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:27.249119 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd82a1-e1f5-45b3-8259-81c6c4c6b1c1 service neutron] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:27.313913 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a24753c-0538-45e8-95e1-f67d12da8813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:27.314689 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24753c-0538-45e8-95e1-f67d12da8813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:27.315047 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24753c-0538-45e8-95e1-f67d12da8813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:27.315416 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24753c-0538-45e8-95e1-f67d12da8813 admin admin] PATH_INFO: `/v3/users/d2ae77b932af41979998646f9e2841c7` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:27.316667 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a24753c-0538-45e8-95e1-f67d12da8813 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2ae77b932af41979998646f9e2841c7)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:27.318788 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a24753c-0538-45e8-95e1-f67d12da8813 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:27.370990 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-4a24753c-0538-45e8-95e1-f67d12da8813 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2ae77b932af41979998646f9e2841c7'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:27.371654 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-4a24753c-0538-45e8-95e1-f67d12da8813 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2ae77b932af41979998646f9e2841c7'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:27.372262 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-4a24753c-0538-45e8-95e1-f67d12da8813 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2ae77b932af41979998646f9e2841c7'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:27.379306 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-4a24753c-0538-45e8-95e1-f67d12da8813 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2ae77b932af41979998646f9e2841c7'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:27.394798 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-4a24753c-0538-45e8-95e1-f67d12da8813 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2ae77b932af41979998646f9e2841c7'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:27.408700 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1222/2469] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:39:27 2022] DELETE /identity/v3/users/d2ae77b932af41979998646f9e2841c7 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:27.871961 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3913831-9196-4a75-8edc-7b1cde031040 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:27.873475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-c3913831-9196-4a75-8edc-7b1cde031040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:27.894598 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3913831-9196-4a75-8edc-7b1cde031040 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:27.987987 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:27.999542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe0f3f-bd77-4211-a600-b207a70568a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:28.000716 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe0f3f-bd77-4211-a600-b207a70568a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:28.001193 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe0f3f-bd77-4211-a600-b207a70568a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:28.001478 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe0f3f-bd77-4211-a600-b207a70568a3 admin admin] PATH_INFO: `/v3/projects/64fd4fa84fe645048c2fae36676e5dd0` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:28.004486 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe0f3f-bd77-4211-a600-b207a70568a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64fd4fa84fe645048c2fae36676e5dd0)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:28.006565 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe0f3f-bd77-4211-a600-b207a70568a3 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:28.057980 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-bebe0f3f-bd77-4211-a600-b207a70568a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64fd4fa84fe645048c2fae36676e5dd0', 'request_id': 'req-bebe0f3f-bd77-4211-a600-b207a70568a3'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:28.064374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-bebe0f3f-bd77-4211-a600-b207a70568a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64fd4fa84fe645048c2fae36676e5dd0', 'request_id': 'req-bebe0f3f-bd77-4211-a600-b207a70568a3'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:28.072485 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-bebe0f3f-bd77-4211-a600-b207a70568a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64fd4fa84fe645048c2fae36676e5dd0', 'request_id': 'req-bebe0f3f-bd77-4211-a600-b207a70568a3'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:28.084703 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-bebe0f3f-bd77-4211-a600-b207a70568a3 admin admin] The token cache is being invalidate because project 64fd4fa84fe645048c2fae36676e5dd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:39:28.085636 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-bebe0f3f-bd77-4211-a600-b207a70568a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:28.086333 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-bebe0f3f-bd77-4211-a600-b207a70568a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:28.087822 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1248/2470] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:39:27 2022] DELETE /identity/v3/projects/64fd4fa84fe645048c2fae36676e5dd0 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:28.093277 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392d7e3-727c-47a8-bbf4-89aafdbbc7e9 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:28.093763 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392d7e3-727c-47a8-bbf4-89aafdbbc7e9 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:28.094183 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392d7e3-727c-47a8-bbf4-89aafdbbc7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:28.116934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-a392d7e3-727c-47a8-bbf4-89aafdbbc7e9 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:28.118315 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-a392d7e3-727c-47a8-bbf4-89aafdbbc7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:28.165254 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:28.187001 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1223/2471] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:28.196246 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf90625-54db-4bb4-a230-ec00bb758b02 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:28.196537 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf90625-54db-4bb4-a230-ec00bb758b02 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:28.196819 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf90625-54db-4bb4-a230-ec00bb758b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:28.219229 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-9cf90625-54db-4bb4-a230-ec00bb758b02 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:28.219229 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-9cf90625-54db-4bb4-a230-ec00bb758b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:28.234532 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1249/2472] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:28.237219 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392d7e3-727c-47a8-bbf4-89aafdbbc7e9 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:28.247453 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392d7e3-727c-47a8-bbf4-89aafdbbc7e9 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:28.252759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8030385e-0ed2-4b2b-a7df-8706d318eff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:28.253478 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030385e-0ed2-4b2b-a7df-8706d318eff2 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:28.253756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030385e-0ed2-4b2b-a7df-8706d318eff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:28.254107 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030385e-0ed2-4b2b-a7df-8706d318eff2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:28.254694 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8030385e-0ed2-4b2b-a7df-8706d318eff2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:39:28.254996 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8030385e-0ed2-4b2b-a7df-8706d318eff2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:28.256610 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8030385e-0ed2-4b2b-a7df-8706d318eff2 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:28.282233 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1224/2473] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:39:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:39:28.286940 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf90625-54db-4bb4-a230-ec00bb758b02 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:28.306941 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf90625-54db-4bb4-a230-ec00bb758b02 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:28.313459 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2f98f-7366-4adc-90f4-30b961b4b3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:28.314337 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2f98f-7366-4adc-90f4-30b961b4b3a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:28.314629 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2f98f-7366-4adc-90f4-30b961b4b3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:28.314907 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2f98f-7366-4adc-90f4-30b961b4b3a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:28.315561 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2f98f-7366-4adc-90f4-30b961b4b3a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:28.317475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2f98f-7366-4adc-90f4-30b961b4b3a3 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:28.371061 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1250/2474] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:39:28 2022] POST /identity/v3/projects => generated 369 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:39:28.373584 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8030385e-0ed2-4b2b-a7df-8706d318eff2 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:28.391234 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8030385e-0ed2-4b2b-a7df-8706d318eff2 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:28.479956 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883cfe5-9314-4605-a807-805d2e331cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:28.480586 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883cfe5-9314-4605-a807-805d2e331cbe admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:28.481138 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883cfe5-9314-4605-a807-805d2e331cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:28.481421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883cfe5-9314-4605-a807-805d2e331cbe admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:28.482083 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7883cfe5-9314-4605-a807-805d2e331cbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:28.484189 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7883cfe5-9314-4605-a807-805d2e331cbe admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:28.515205 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1225/2475] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:39:28 2022] POST /identity/v3/users => generated 285 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:39:28.516859 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2f98f-7366-4adc-90f4-30b961b4b3a3 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:28.529208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2f98f-7366-4adc-90f4-30b961b4b3a3 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:28.533894 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55860273-2dcb-402c-a25c-034c962026f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:28.534524 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55860273-2dcb-402c-a25c-034c962026f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:28.534799 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55860273-2dcb-402c-a25c-034c962026f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:28.535071 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55860273-2dcb-402c-a25c-034c962026f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:28.535706 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55860273-2dcb-402c-a25c-034c962026f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:28.537293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55860273-2dcb-402c-a25c-034c962026f2 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:28.554390 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1251/2476] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:39:28.555972 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883cfe5-9314-4605-a807-805d2e331cbe admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:28.568711 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883cfe5-9314-4605-a807-805d2e331cbe admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:28.573580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0395215-1f9d-42e8-90b7-06cb06b711cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:28.574277 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0395215-1f9d-42e8-90b7-06cb06b711cc admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:28.574622 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0395215-1f9d-42e8-90b7-06cb06b711cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:28.574984 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0395215-1f9d-42e8-90b7-06cb06b711cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:28.575691 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0395215-1f9d-42e8-90b7-06cb06b711cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:28.577342 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0395215-1f9d-42e8-90b7-06cb06b711cc admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:28.588269 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1226/2477] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:39:28.592429 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55860273-2dcb-402c-a25c-034c962026f2 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:28.603004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55860273-2dcb-402c-a25c-034c962026f2 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:28.608379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff0fc3d-aceb-4348-aba9-9618df5b6108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:28.609105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0fc3d-aceb-4348-aba9-9618df5b6108 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:28.609453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0fc3d-aceb-4348-aba9-9618df5b6108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:28.609812 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0fc3d-aceb-4348-aba9-9618df5b6108 admin admin] PATH_INFO: `/v3/projects/cd6bd0015b874e6da5e5d52de6b14a1a/users/a6325e2cfc3c4b31937a150252139261/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:28.632116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff0fc3d-aceb-4348-aba9-9618df5b6108 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd6bd0015b874e6da5e5d52de6b14a1a, user_id=a6325e2cfc3c4b31937a150252139261, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:28.634935 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff0fc3d-aceb-4348-aba9-9618df5b6108 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:28.650301 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1252/2478] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:39:28 2022] PUT /identity/v3/projects/cd6bd0015b874e6da5e5d52de6b14a1a/users/a6325e2cfc3c4b31937a150252139261/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:39:28.660873 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18460f-541f-42bc-b566-a7a86afdd19d None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:28.661156 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18460f-541f-42bc-b566-a7a86afdd19d None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:28.661444 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18460f-541f-42bc-b566-a7a86afdd19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:28.691224 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-1f18460f-541f-42bc-b566-a7a86afdd19d None None] MFA Rules not processed for user `a6325e2cfc3c4b31937a150252139261`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:28.692539 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-1f18460f-541f-42bc-b566-a7a86afdd19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:28.763519 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1227/2479] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:28.768049 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff0fc3d-aceb-4348-aba9-9618df5b6108 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:28.852717 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff0fc3d-aceb-4348-aba9-9618df5b6108 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:28.857996 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b173c1bc-9027-4787-9d7f-0dc8d6cdf9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:28.858642 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173c1bc-9027-4787-9d7f-0dc8d6cdf9de admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:28.859403 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173c1bc-9027-4787-9d7f-0dc8d6cdf9de admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:28.859403 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173c1bc-9027-4787-9d7f-0dc8d6cdf9de admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:28.859952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b173c1bc-9027-4787-9d7f-0dc8d6cdf9de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:28.861908 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b173c1bc-9027-4787-9d7f-0dc8d6cdf9de admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:28.895714 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1253/2480] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:39:28 2022] POST /identity/v3/projects => generated 367 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:39:28.900093 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f18460f-541f-42bc-b566-a7a86afdd19d None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:28.915015 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f18460f-541f-42bc-b566-a7a86afdd19d None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:28.966439 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:29.000059 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb28905-1453-464a-bd51-0fa10ad958c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:29.000684 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb28905-1453-464a-bd51-0fa10ad958c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:29.000969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb28905-1453-464a-bd51-0fa10ad958c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:29.001245 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb28905-1453-464a-bd51-0fa10ad958c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:29.001884 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb28905-1453-464a-bd51-0fa10ad958c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:29.003861 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb28905-1453-464a-bd51-0fa10ad958c8 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:29.030868 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1228/2481] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:39:28 2022] POST /identity/v3/users => generated 290 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:39:29.032109 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b173c1bc-9027-4787-9d7f-0dc8d6cdf9de admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:29.040600 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:29.045672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b173c1bc-9027-4787-9d7f-0dc8d6cdf9de admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:29.059440 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e578c92-bfdf-4ed1-a6d0-daaf1e559913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:29.060055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e578c92-bfdf-4ed1-a6d0-daaf1e559913 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:29.060563 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e578c92-bfdf-4ed1-a6d0-daaf1e559913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:29.060997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e578c92-bfdf-4ed1-a6d0-daaf1e559913 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:29.061653 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e578c92-bfdf-4ed1-a6d0-daaf1e559913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:29.063300 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e578c92-bfdf-4ed1-a6d0-daaf1e559913 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:29.083246 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1254/2482] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:39:29.087972 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb28905-1453-464a-bd51-0fa10ad958c8 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:29.099586 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb28905-1453-464a-bd51-0fa10ad958c8 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:29.104952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8d02eb-7b6a-48d7-af41-a195caccd2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:29.105600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d02eb-7b6a-48d7-af41-a195caccd2d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:29.106182 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d02eb-7b6a-48d7-af41-a195caccd2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:29.106451 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d02eb-7b6a-48d7-af41-a195caccd2d6 admin admin] PATH_INFO: `/v3/projects/8fa95e01469f434e8a07134efdb723c9/users/d1e0bffda0ee4b2598e263c2300e938d/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:29.128520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8d02eb-7b6a-48d7-af41-a195caccd2d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fa95e01469f434e8a07134efdb723c9, user_id=d1e0bffda0ee4b2598e263c2300e938d, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:29.131445 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8d02eb-7b6a-48d7-af41-a195caccd2d6 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:29.147430 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1229/2483] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:39:29 2022] PUT /identity/v3/projects/8fa95e01469f434e8a07134efdb723c9/users/d1e0bffda0ee4b2598e263c2300e938d/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:39:29.152193 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e578c92-bfdf-4ed1-a6d0-daaf1e559913 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:29.162946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e578c92-bfdf-4ed1-a6d0-daaf1e559913 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:29.249471 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b624919-a08a-4c4b-8906-942415755c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:29.249471 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b624919-a08a-4c4b-8906-942415755c96 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:29.249471 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b624919-a08a-4c4b-8906-942415755c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:29.249471 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b624919-a08a-4c4b-8906-942415755c96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:29.249946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b624919-a08a-4c4b-8906-942415755c96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:29.251260 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b624919-a08a-4c4b-8906-942415755c96 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:29.271497 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1255/2484] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:39:29.276453 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8d02eb-7b6a-48d7-af41-a195caccd2d6 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:29.295841 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8d02eb-7b6a-48d7-af41-a195caccd2d6 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:29.302828 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ad2fc-7333-4508-8f61-a7947c535be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:29.303065 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ad2fc-7333-4508-8f61-a7947c535be5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:29.303511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ad2fc-7333-4508-8f61-a7947c535be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:29.303511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ad2fc-7333-4508-8f61-a7947c535be5 admin admin] PATH_INFO: `/v3/projects/8fa95e01469f434e8a07134efdb723c9/users/d1e0bffda0ee4b2598e263c2300e938d/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:29.305713 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7ad2fc-7333-4508-8f61-a7947c535be5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fa95e01469f434e8a07134efdb723c9, user_id=d1e0bffda0ee4b2598e263c2300e938d, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:29.309094 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7ad2fc-7333-4508-8f61-a7947c535be5 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:29.321292 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1230/2485] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:39:29 2022] PUT /identity/v3/projects/8fa95e01469f434e8a07134efdb723c9/users/d1e0bffda0ee4b2598e263c2300e938d/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:29.333424 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefdc70-fa68-437b-8cad-1e0fef407038 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:29.333716 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefdc70-fa68-437b-8cad-1e0fef407038 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:29.333991 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefdc70-fa68-437b-8cad-1e0fef407038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:29.375692 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-8aefdc70-fa68-437b-8cad-1e0fef407038 None None] MFA Rules not processed for user `d1e0bffda0ee4b2598e263c2300e938d`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:29.377382 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-8aefdc70-fa68-437b-8cad-1e0fef407038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:29.462143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1256/2486] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:29 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:29.477432 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ad2fc-7333-4508-8f61-a7947c535be5 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:29.479217 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-ed7ad2fc-7333-4508-8f61-a7947c535be5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:29.543076 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ad2fc-7333-4508-8f61-a7947c535be5 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:29.546941 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a099849-4529-4e80-9b8b-a1c2156f752c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:29.547546 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a099849-4529-4e80-9b8b-a1c2156f752c service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:29.547829 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a099849-4529-4e80-9b8b-a1c2156f752c service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:29.548099 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a099849-4529-4e80-9b8b-a1c2156f752c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:29.557699 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a099849-4529-4e80-9b8b-a1c2156f752c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:29.559156 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a099849-4529-4e80-9b8b-a1c2156f752c service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:29.569240 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1231/2487] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:39:29 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:39:29.620518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aefdc70-fa68-437b-8cad-1e0fef407038 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:29.622371 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-8aefdc70-fa68-437b-8cad-1e0fef407038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:29.639119 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aefdc70-fa68-437b-8cad-1e0fef407038 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:29.642924 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5844c-06cb-4625-9282-95626020ef27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:29.643534 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5844c-06cb-4625-9282-95626020ef27 service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:29.643811 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5844c-06cb-4625-9282-95626020ef27 service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:29.644081 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5844c-06cb-4625-9282-95626020ef27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:29.710701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f5844c-06cb-4625-9282-95626020ef27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:29.712057 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f5844c-06cb-4625-9282-95626020ef27 service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:29.721715 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1257/2488] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:39:29 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:39:33.106347 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a099849-4529-4e80-9b8b-a1c2156f752c service cinder] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:33.170868 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a099849-4529-4e80-9b8b-a1c2156f752c service cinder] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:33.175734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da4a45b-f3e8-4354-ab40-8b25a2127c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:33.176372 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4a45b-f3e8-4354-ab40-8b25a2127c45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:33.176653 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4a45b-f3e8-4354-ab40-8b25a2127c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:33.176921 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4a45b-f3e8-4354-ab40-8b25a2127c45 admin admin] PATH_INFO: `/v3/users/a6325e2cfc3c4b31937a150252139261` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:33.178108 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da4a45b-f3e8-4354-ab40-8b25a2127c45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6325e2cfc3c4b31937a150252139261)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:33.180788 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da4a45b-f3e8-4354-ab40-8b25a2127c45 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:33.222653 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:33.230058 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6da4a45b-f3e8-4354-ab40-8b25a2127c45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6325e2cfc3c4b31937a150252139261'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:33.231231 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6da4a45b-f3e8-4354-ab40-8b25a2127c45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6325e2cfc3c4b31937a150252139261'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:33.232040 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6da4a45b-f3e8-4354-ab40-8b25a2127c45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6325e2cfc3c4b31937a150252139261'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:33.239102 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6da4a45b-f3e8-4354-ab40-8b25a2127c45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6325e2cfc3c4b31937a150252139261'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:33.251715 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6da4a45b-f3e8-4354-ab40-8b25a2127c45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6325e2cfc3c4b31937a150252139261'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:33.264043 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1232/2489] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:39:33 2022] DELETE /identity/v3/users/a6325e2cfc3c4b31937a150252139261 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:39:33.268544 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0f781-21f4-455e-ba0d-de3fcac16267 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:33.269014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0f781-21f4-455e-ba0d-de3fcac16267 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:33.269129 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0f781-21f4-455e-ba0d-de3fcac16267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:33.288709 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-79d0f781-21f4-455e-ba0d-de3fcac16267 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:33.289922 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-79d0f781-21f4-455e-ba0d-de3fcac16267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:33.298000 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=100 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:33.348666 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1258/2490] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:33.361562 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da4a45b-f3e8-4354-ab40-8b25a2127c45 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:33.362983 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-6da4a45b-f3e8-4354-ab40-8b25a2127c45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:33.421637 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da4a45b-f3e8-4354-ab40-8b25a2127c45 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:33.425376 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de85a16-fc88-49a0-aaa3-79f90995ff1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:33.425979 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de85a16-fc88-49a0-aaa3-79f90995ff1b service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:33.426281 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de85a16-fc88-49a0-aaa3-79f90995ff1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:33.426556 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de85a16-fc88-49a0-aaa3-79f90995ff1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:33.435557 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de85a16-fc88-49a0-aaa3-79f90995ff1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:33.436976 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de85a16-fc88-49a0-aaa3-79f90995ff1b service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:33.446418 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1233/2491] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:39:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:39:33.691647 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0f781-21f4-455e-ba0d-de3fcac16267 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:33.701906 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0f781-21f4-455e-ba0d-de3fcac16267 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:33.706425 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a62e5d-ffd5-4b7b-860e-2c14635ee1c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:33.707153 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a62e5d-ffd5-4b7b-860e-2c14635ee1c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:33.707451 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a62e5d-ffd5-4b7b-860e-2c14635ee1c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:33.707901 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a62e5d-ffd5-4b7b-860e-2c14635ee1c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:33.709774 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-12a62e5d-ffd5-4b7b-860e-2c14635ee1c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:33.722529 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de85a16-fc88-49a0-aaa3-79f90995ff1b service neutron] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:33.724147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-3de85a16-fc88-49a0-aaa3-79f90995ff1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:33.724793 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a62e5d-ffd5-4b7b-860e-2c14635ee1c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:33.726608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a62e5d-ffd5-4b7b-860e-2c14635ee1c1 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:33.737360 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de85a16-fc88-49a0-aaa3-79f90995ff1b service neutron] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:33.741066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1259/2492] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:39:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:39:33.742439 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08f9bb-cb91-4fcd-8254-34a0c4291529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:33.743123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08f9bb-cb91-4fcd-8254-34a0c4291529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:33.743402 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08f9bb-cb91-4fcd-8254-34a0c4291529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:33.743744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08f9bb-cb91-4fcd-8254-34a0c4291529 admin admin] PATH_INFO: `/v3/projects/cd6bd0015b874e6da5e5d52de6b14a1a` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:33.745026 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08f9bb-cb91-4fcd-8254-34a0c4291529 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd6bd0015b874e6da5e5d52de6b14a1a)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:33.747032 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08f9bb-cb91-4fcd-8254-34a0c4291529 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:33.807411 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6a08f9bb-cb91-4fcd-8254-34a0c4291529 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd6bd0015b874e6da5e5d52de6b14a1a', 'request_id': 'req-6a08f9bb-cb91-4fcd-8254-34a0c4291529'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:33.813826 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6a08f9bb-cb91-4fcd-8254-34a0c4291529 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd6bd0015b874e6da5e5d52de6b14a1a', 'request_id': 'req-6a08f9bb-cb91-4fcd-8254-34a0c4291529'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:33.820145 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6a08f9bb-cb91-4fcd-8254-34a0c4291529 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd6bd0015b874e6da5e5d52de6b14a1a', 'request_id': 'req-6a08f9bb-cb91-4fcd-8254-34a0c4291529'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:33.833401 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6a08f9bb-cb91-4fcd-8254-34a0c4291529 admin admin] The token cache is being invalidate because project cd6bd0015b874e6da5e5d52de6b14a1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:39:33.833685 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6a08f9bb-cb91-4fcd-8254-34a0c4291529 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:33.834344 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6a08f9bb-cb91-4fcd-8254-34a0c4291529 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:33.836570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1234/2493] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:39:33 2022] DELETE /identity/v3/projects/cd6bd0015b874e6da5e5d52de6b14a1a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:39:33.840598 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a62e5d-ffd5-4b7b-860e-2c14635ee1c1 service neutron] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:33.850516 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a62e5d-ffd5-4b7b-860e-2c14635ee1c1 service neutron] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:33.913175 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d9fde4-4f70-474e-a720-92ac0719ada0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:33.913815 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9fde4-4f70-474e-a720-92ac0719ada0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:33.914120 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9fde4-4f70-474e-a720-92ac0719ada0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:33.914389 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9fde4-4f70-474e-a720-92ac0719ada0 admin admin] PATH_INFO: `/v3/users/d1e0bffda0ee4b2598e263c2300e938d` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:33.915522 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d9fde4-4f70-474e-a720-92ac0719ada0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1e0bffda0ee4b2598e263c2300e938d)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:33.917415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d9fde4-4f70-474e-a720-92ac0719ada0 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:33.967042 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-97d9fde4-4f70-474e-a720-92ac0719ada0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1e0bffda0ee4b2598e263c2300e938d'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:33.967675 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-97d9fde4-4f70-474e-a720-92ac0719ada0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1e0bffda0ee4b2598e263c2300e938d'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:33.974934 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-97d9fde4-4f70-474e-a720-92ac0719ada0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1e0bffda0ee4b2598e263c2300e938d'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:33.987623 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-97d9fde4-4f70-474e-a720-92ac0719ada0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1e0bffda0ee4b2598e263c2300e938d'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:33.988328 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-97d9fde4-4f70-474e-a720-92ac0719ada0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1e0bffda0ee4b2598e263c2300e938d'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:34.001453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1260/2494] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:39:33 2022] DELETE /identity/v3/users/d1e0bffda0ee4b2598e263c2300e938d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:34.285291 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08f9bb-cb91-4fcd-8254-34a0c4291529 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:34.286756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-6a08f9bb-cb91-4fcd-8254-34a0c4291529 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:34.301408 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08f9bb-cb91-4fcd-8254-34a0c4291529 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:34.345691 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:34.364415 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9eab5-4567-46a9-8674-a023a45684d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:34.365058 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9eab5-4567-46a9-8674-a023a45684d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:34.365340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9eab5-4567-46a9-8674-a023a45684d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:34.365611 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9eab5-4567-46a9-8674-a023a45684d6 admin admin] PATH_INFO: `/v3/projects/8fa95e01469f434e8a07134efdb723c9` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:34.366853 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9eab5-4567-46a9-8674-a023a45684d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fa95e01469f434e8a07134efdb723c9)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:34.368796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9eab5-4567-46a9-8674-a023a45684d6 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:34.418113 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-48a9eab5-4567-46a9-8674-a023a45684d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fa95e01469f434e8a07134efdb723c9', 'request_id': 'req-48a9eab5-4567-46a9-8674-a023a45684d6'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:34.424828 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-48a9eab5-4567-46a9-8674-a023a45684d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fa95e01469f434e8a07134efdb723c9', 'request_id': 'req-48a9eab5-4567-46a9-8674-a023a45684d6'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:34.430580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-48a9eab5-4567-46a9-8674-a023a45684d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fa95e01469f434e8a07134efdb723c9', 'request_id': 'req-48a9eab5-4567-46a9-8674-a023a45684d6'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:34.444083 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-48a9eab5-4567-46a9-8674-a023a45684d6 admin admin] The token cache is being invalidate because project 8fa95e01469f434e8a07134efdb723c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:39:34.445012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-48a9eab5-4567-46a9-8674-a023a45684d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:34.445732 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-48a9eab5-4567-46a9-8674-a023a45684d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:34.447678 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1235/2495] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:39:34 2022] DELETE /identity/v3/projects/8fa95e01469f434e8a07134efdb723c9 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:34.452285 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf72b0b-bfe3-4ee4-817b-fdf0dfafee72 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:34.452787 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf72b0b-bfe3-4ee4-817b-fdf0dfafee72 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:34.452884 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf72b0b-bfe3-4ee4-817b-fdf0dfafee72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:34.473669 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-4cf72b0b-bfe3-4ee4-817b-fdf0dfafee72 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:34.474943 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-4cf72b0b-bfe3-4ee4-817b-fdf0dfafee72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:34.489259 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:34.539728 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1261/2496] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:34.552834 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686596d-4cfd-4791-8be4-49243e2b1227 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:34.553115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686596d-4cfd-4791-8be4-49243e2b1227 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:34.553379 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686596d-4cfd-4791-8be4-49243e2b1227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:34.574997 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-7686596d-4cfd-4791-8be4-49243e2b1227 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:34.576235 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-7686596d-4cfd-4791-8be4-49243e2b1227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:34.587367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1236/2497] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:34.591766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf72b0b-bfe3-4ee4-817b-fdf0dfafee72 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:34.601949 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf72b0b-bfe3-4ee4-817b-fdf0dfafee72 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:34.607402 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5266ed6-760b-451e-a41a-beed7fb1364d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:34.608045 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5266ed6-760b-451e-a41a-beed7fb1364d admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:34.608328 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5266ed6-760b-451e-a41a-beed7fb1364d admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:34.608609 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5266ed6-760b-451e-a41a-beed7fb1364d admin admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:34.609642 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5266ed6-760b-451e-a41a-beed7fb1364d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:39:34.609889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5266ed6-760b-451e-a41a-beed7fb1364d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:34.611686 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5266ed6-760b-451e-a41a-beed7fb1364d admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:34.625628 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1262/2498] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:39:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:39:34.629935 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686596d-4cfd-4791-8be4-49243e2b1227 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:34.639444 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686596d-4cfd-4791-8be4-49243e2b1227 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:34.644538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eef6d8-659c-49f4-9274-4c25aa7b7dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:34.645219 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eef6d8-659c-49f4-9274-4c25aa7b7dbb admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:34.645597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eef6d8-659c-49f4-9274-4c25aa7b7dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:34.645907 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eef6d8-659c-49f4-9274-4c25aa7b7dbb admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:34.647097 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eef6d8-659c-49f4-9274-4c25aa7b7dbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:34.648728 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eef6d8-659c-49f4-9274-4c25aa7b7dbb admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:34.684300 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1237/2499] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:39:34 2022] POST /identity/v3/projects => generated 399 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:39:34.688992 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5266ed6-760b-451e-a41a-beed7fb1364d admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:34.699344 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5266ed6-760b-451e-a41a-beed7fb1364d admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:34.762299 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f957ad0a-cd8f-4dd0-b2d9-70453f458bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:34.763013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957ad0a-cd8f-4dd0-b2d9-70453f458bd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:34.763365 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957ad0a-cd8f-4dd0-b2d9-70453f458bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:34.763698 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957ad0a-cd8f-4dd0-b2d9-70453f458bd3 admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:34.764404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f957ad0a-cd8f-4dd0-b2d9-70453f458bd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:34.766456 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f957ad0a-cd8f-4dd0-b2d9-70453f458bd3 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:34.791074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1263/2500] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:39:34 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:39:34.799968 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eef6d8-659c-49f4-9274-4c25aa7b7dbb admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:34.817593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eef6d8-659c-49f4-9274-4c25aa7b7dbb admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:34.822504 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c2e31-df4c-47b6-99b4-e400cdb76223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:34.823106 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c2e31-df4c-47b6-99b4-e400cdb76223 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:34.823383 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c2e31-df4c-47b6-99b4-e400cdb76223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:34.823650 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86c2e31-df4c-47b6-99b4-e400cdb76223 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:34.824387 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86c2e31-df4c-47b6-99b4-e400cdb76223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:34.825863 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86c2e31-df4c-47b6-99b4-e400cdb76223 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:34.840447 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1238/2501] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:39:34.841626 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f957ad0a-cd8f-4dd0-b2d9-70453f458bd3 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:34.851788 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f957ad0a-cd8f-4dd0-b2d9-70453f458bd3 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:34.856652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1aa15f-a555-4558-8e6b-5bc201670ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:34.857240 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1aa15f-a555-4558-8e6b-5bc201670ae0 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:34.857515 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1aa15f-a555-4558-8e6b-5bc201670ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:34.857783 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1aa15f-a555-4558-8e6b-5bc201670ae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:34.858451 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1aa15f-a555-4558-8e6b-5bc201670ae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:34.860018 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1aa15f-a555-4558-8e6b-5bc201670ae0 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:34.871197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1264/2502] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:39:34.875807 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c2e31-df4c-47b6-99b4-e400cdb76223 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:34.886649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86c2e31-df4c-47b6-99b4-e400cdb76223 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:34.891257 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5fb410-3ecb-43cf-9e0e-d36d8ec074de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:34.891907 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fb410-3ecb-43cf-9e0e-d36d8ec074de admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:34.892184 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fb410-3ecb-43cf-9e0e-d36d8ec074de admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:34.892457 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fb410-3ecb-43cf-9e0e-d36d8ec074de admin admin] PATH_INFO: `/v3/projects/eee3161e2ada49c5b144b913813e8bf6/users/b9bb121f42dc427384c1827ca4f6b58c/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:34.907265 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5fb410-3ecb-43cf-9e0e-d36d8ec074de admin admin] RBAC: Authorizing `identity:create_grant(project_id=eee3161e2ada49c5b144b913813e8bf6, user_id=b9bb121f42dc427384c1827ca4f6b58c, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:34.909939 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5fb410-3ecb-43cf-9e0e-d36d8ec074de admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:34.924088 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1239/2503] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:39:34 2022] PUT /identity/v3/projects/eee3161e2ada49c5b144b913813e8bf6/users/b9bb121f42dc427384c1827ca4f6b58c/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:34.934926 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda8e54-ec0e-4e75-9934-05b29a742078 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:34.935335 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda8e54-ec0e-4e75-9934-05b29a742078 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:34.935615 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda8e54-ec0e-4e75-9934-05b29a742078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:34.964236 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-8eda8e54-ec0e-4e75-9934-05b29a742078 None None] MFA Rules not processed for user `b9bb121f42dc427384c1827ca4f6b58c`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:34.965478 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-8eda8e54-ec0e-4e75-9934-05b29a742078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:35.031073 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1265/2504] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:34 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:35.035522 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5fb410-3ecb-43cf-9e0e-d36d8ec074de admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:35.098151 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5fb410-3ecb-43cf-9e0e-d36d8ec074de admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:35.103067 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6613e-b095-41a9-ab12-fab155661d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:35.103670 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6613e-b095-41a9-ab12-fab155661d31 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:35.103947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6613e-b095-41a9-ab12-fab155661d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:35.104217 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6613e-b095-41a9-ab12-fab155661d31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:35.104856 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6613e-b095-41a9-ab12-fab155661d31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:35.107053 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6613e-b095-41a9-ab12-fab155661d31 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:35.122473 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:35.139409 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1240/2505] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:39:35 2022] POST /identity/v3/projects => generated 401 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:39:35.145157 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda8e54-ec0e-4e75-9934-05b29a742078 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:35.157516 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda8e54-ec0e-4e75-9934-05b29a742078 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:35.170432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:35.223290 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3525b2-0a0b-42e9-8b0b-21617d37a32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:35.224026 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3525b2-0a0b-42e9-8b0b-21617d37a32c admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:35.224391 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3525b2-0a0b-42e9-8b0b-21617d37a32c admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:35.224732 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3525b2-0a0b-42e9-8b0b-21617d37a32c admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:35.225440 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3525b2-0a0b-42e9-8b0b-21617d37a32c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:35.228059 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3525b2-0a0b-42e9-8b0b-21617d37a32c admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:35.255790 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1266/2506] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:39:35 2022] POST /identity/v3/users => generated 307 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:39:35.257191 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6613e-b095-41a9-ab12-fab155661d31 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:35.268538 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6613e-b095-41a9-ab12-fab155661d31 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:35.273751 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693903d-dc50-48fb-9647-66360c50efb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:35.274549 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693903d-dc50-48fb-9647-66360c50efb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:35.274922 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693903d-dc50-48fb-9647-66360c50efb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:35.275293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693903d-dc50-48fb-9647-66360c50efb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:35.276038 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f693903d-dc50-48fb-9647-66360c50efb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:35.278202 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f693903d-dc50-48fb-9647-66360c50efb3 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:35.312704 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3525b2-0a0b-42e9-8b0b-21617d37a32c admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:35.313532 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1241/2507] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:39:35.323963 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3525b2-0a0b-42e9-8b0b-21617d37a32c admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:35.329164 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df048644-b7c0-404b-b6ad-f4bf9449547b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:35.329991 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df048644-b7c0-404b-b6ad-f4bf9449547b admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:35.330224 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df048644-b7c0-404b-b6ad-f4bf9449547b admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:35.330506 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df048644-b7c0-404b-b6ad-f4bf9449547b admin admin] PATH_INFO: `/v3/projects/612f0044fcff4d8cb1af1627446a6834/users/1d55bc257c1d4a16ae55f4844f961403/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:35.352063 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df048644-b7c0-404b-b6ad-f4bf9449547b admin admin] RBAC: Authorizing `identity:create_grant(project_id=612f0044fcff4d8cb1af1627446a6834, user_id=1d55bc257c1d4a16ae55f4844f961403, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:35.355224 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df048644-b7c0-404b-b6ad-f4bf9449547b admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:35.373217 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1267/2508] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:39:35 2022] PUT /identity/v3/projects/612f0044fcff4d8cb1af1627446a6834/users/1d55bc257c1d4a16ae55f4844f961403/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:39:35.377113 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693903d-dc50-48fb-9647-66360c50efb3 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:35.388970 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693903d-dc50-48fb-9647-66360c50efb3 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:35.461501 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120452be-2a29-459b-9392-4fdea6f64dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:35.462291 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120452be-2a29-459b-9392-4fdea6f64dc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:35.462637 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120452be-2a29-459b-9392-4fdea6f64dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:35.462978 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120452be-2a29-459b-9392-4fdea6f64dc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:35.463719 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120452be-2a29-459b-9392-4fdea6f64dc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:35.465555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120452be-2a29-459b-9392-4fdea6f64dc5 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:35.479045 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1242/2509] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:39:35.484933 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df048644-b7c0-404b-b6ad-f4bf9449547b admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:35.501117 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df048644-b7c0-404b-b6ad-f4bf9449547b admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:35.508365 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92287b-cde9-4c1f-93b7-f030d8bc1482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:35.509030 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92287b-cde9-4c1f-93b7-f030d8bc1482 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:35.509307 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92287b-cde9-4c1f-93b7-f030d8bc1482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:35.509575 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92287b-cde9-4c1f-93b7-f030d8bc1482 admin admin] PATH_INFO: `/v3/projects/612f0044fcff4d8cb1af1627446a6834/users/1d55bc257c1d4a16ae55f4844f961403/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:35.511731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df92287b-cde9-4c1f-93b7-f030d8bc1482 admin admin] RBAC: Authorizing `identity:create_grant(project_id=612f0044fcff4d8cb1af1627446a6834, user_id=1d55bc257c1d4a16ae55f4844f961403, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:35.514570 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df92287b-cde9-4c1f-93b7-f030d8bc1482 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:35.539450 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1268/2510] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:39:35 2022] PUT /identity/v3/projects/612f0044fcff4d8cb1af1627446a6834/users/1d55bc257c1d4a16ae55f4844f961403/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:35.541743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f5c09-bdf2-4397-84d0-68ccde7039ed None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:35.542054 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f5c09-bdf2-4397-84d0-68ccde7039ed None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:35.542361 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f5c09-bdf2-4397-84d0-68ccde7039ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:35.573071 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-fb3f5c09-bdf2-4397-84d0-68ccde7039ed None None] MFA Rules not processed for user `1d55bc257c1d4a16ae55f4844f961403`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:35.574365 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-fb3f5c09-bdf2-4397-84d0-68ccde7039ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:35.661690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1243/2511] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:35 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:35.670667 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92287b-cde9-4c1f-93b7-f030d8bc1482 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:35.672550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-df92287b-cde9-4c1f-93b7-f030d8bc1482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:35.738856 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92287b-cde9-4c1f-93b7-f030d8bc1482 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:35.743419 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0701d2-2a3c-41fb-92a5-e911c73552b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:35.744568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0701d2-2a3c-41fb-92a5-e911c73552b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:35.745387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0701d2-2a3c-41fb-92a5-e911c73552b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:35.745387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0701d2-2a3c-41fb-92a5-e911c73552b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:35.759063 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0701d2-2a3c-41fb-92a5-e911c73552b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:35.760544 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0701d2-2a3c-41fb-92a5-e911c73552b6 service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:35.772316 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1269/2512] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:39:35 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:39:35.867004 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f5c09-bdf2-4397-84d0-68ccde7039ed None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:35.878276 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f5c09-bdf2-4397-84d0-68ccde7039ed None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:35.883277 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc14511-4a78-408f-b728-a97c4b421116 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:35.883906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc14511-4a78-408f-b728-a97c4b421116 service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:35.884181 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc14511-4a78-408f-b728-a97c4b421116 service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:35.884453 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc14511-4a78-408f-b728-a97c4b421116 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:35.961900 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc14511-4a78-408f-b728-a97c4b421116 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:35.963411 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc14511-4a78-408f-b728-a97c4b421116 service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:35.973430 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1244/2513] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:39:35 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:39:37.109020 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0701d2-2a3c-41fb-92a5-e911c73552b6 service cinder] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:37.110999 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-4e0701d2-2a3c-41fb-92a5-e911c73552b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:37.198219 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0701d2-2a3c-41fb-92a5-e911c73552b6 service cinder] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:37.208968 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029de62f-7367-4d34-a6e5-0c2644465a61 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84b02e7351914cabb67462454b328d05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:37.210040 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029de62f-7367-4d34-a6e5-0c2644465a61 service placement] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:37.210638 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029de62f-7367-4d34-a6e5-0c2644465a61 service placement] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:37.211032 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029de62f-7367-4d34-a6e5-0c2644465a61 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:37.213866 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-029de62f-7367-4d34-a6e5-0c2644465a61 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:37.227666 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029de62f-7367-4d34-a6e5-0c2644465a61 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:37.229271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029de62f-7367-4d34-a6e5-0c2644465a61 service placement] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:37.236392 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:37.242664 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1270/2514] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:39:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:39:37.417157 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc14511-4a78-408f-b728-a97c4b421116 service cinder] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:37.418715 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-3bc14511-4a78-408f-b728-a97c4b421116 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:37.424109 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:37.492146 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc14511-4a78-408f-b728-a97c4b421116 service cinder] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:37.498330 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb791de-cf25-4cd0-be16-607e6bf0e607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:37.498956 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb791de-cf25-4cd0-be16-607e6bf0e607 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:37.499238 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb791de-cf25-4cd0-be16-607e6bf0e607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:37.499500 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb791de-cf25-4cd0-be16-607e6bf0e607 admin admin] PATH_INFO: `/v3/users/b486a094550541729c19c8713a648243` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:37.500624 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb791de-cf25-4cd0-be16-607e6bf0e607 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b486a094550541729c19c8713a648243)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:37.502594 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb791de-cf25-4cd0-be16-607e6bf0e607 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:37.552684 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-deb791de-cf25-4cd0-be16-607e6bf0e607 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b486a094550541729c19c8713a648243'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:37.553344 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-deb791de-cf25-4cd0-be16-607e6bf0e607 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b486a094550541729c19c8713a648243'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:37.554010 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-deb791de-cf25-4cd0-be16-607e6bf0e607 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b486a094550541729c19c8713a648243'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:37.561036 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-deb791de-cf25-4cd0-be16-607e6bf0e607 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b486a094550541729c19c8713a648243'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:37.573707 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-deb791de-cf25-4cd0-be16-607e6bf0e607 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b486a094550541729c19c8713a648243'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:37.590678 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1245/2515] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:39:37 2022] DELETE /identity/v3/users/b486a094550541729c19c8713a648243 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:37.766268 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029de62f-7367-4d34-a6e5-0c2644465a61 service placement] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:37.779751 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029de62f-7367-4d34-a6e5-0c2644465a61 service placement] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:37.867619 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e21cb-9f38-42e4-8302-caaec95b8628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:37.868472 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e21cb-9f38-42e4-8302-caaec95b8628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:37.868905 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e21cb-9f38-42e4-8302-caaec95b8628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:37.869185 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e21cb-9f38-42e4-8302-caaec95b8628 admin admin] PATH_INFO: `/v3/projects/49df2764440f4f2cb68d8d6284f07f47` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:37.870361 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0e21cb-9f38-42e4-8302-caaec95b8628 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49df2764440f4f2cb68d8d6284f07f47)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:37.872266 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0e21cb-9f38-42e4-8302-caaec95b8628 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:37.931362 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-5d0e21cb-9f38-42e4-8302-caaec95b8628 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49df2764440f4f2cb68d8d6284f07f47', 'request_id': 'req-5d0e21cb-9f38-42e4-8302-caaec95b8628'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:37.937503 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-5d0e21cb-9f38-42e4-8302-caaec95b8628 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49df2764440f4f2cb68d8d6284f07f47', 'request_id': 'req-5d0e21cb-9f38-42e4-8302-caaec95b8628'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:37.945881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-5d0e21cb-9f38-42e4-8302-caaec95b8628 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49df2764440f4f2cb68d8d6284f07f47', 'request_id': 'req-5d0e21cb-9f38-42e4-8302-caaec95b8628'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:37.956881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-5d0e21cb-9f38-42e4-8302-caaec95b8628 admin admin] The token cache is being invalidate because project 49df2764440f4f2cb68d8d6284f07f47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:39:37.957172 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-5d0e21cb-9f38-42e4-8302-caaec95b8628 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:37.957832 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-5d0e21cb-9f38-42e4-8302-caaec95b8628 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:37.959367 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1271/2516] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:39:37 2022] DELETE /identity/v3/projects/49df2764440f4f2cb68d8d6284f07f47 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:37.964445 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b596c-5bc2-4294-a064-3f8d5c6877b9 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:37.964719 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b596c-5bc2-4294-a064-3f8d5c6877b9 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:37.964978 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b596c-5bc2-4294-a064-3f8d5c6877b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:37.986443 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-6d2b596c-5bc2-4294-a064-3f8d5c6877b9 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:37.987687 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-6d2b596c-5bc2-4294-a064-3f8d5c6877b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:38.052887 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1246/2517] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:38.061601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60609fe0-4d05-479a-a6bc-bc10c1397b10 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:38.061975 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60609fe0-4d05-479a-a6bc-bc10c1397b10 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:38.062300 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60609fe0-4d05-479a-a6bc-bc10c1397b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:38.083204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-60609fe0-4d05-479a-a6bc-bc10c1397b10 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:38.084504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-60609fe0-4d05-479a-a6bc-bc10c1397b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:38.093035 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1272/2518] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:38.097410 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b596c-5bc2-4294-a064-3f8d5c6877b9 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:38.107721 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b596c-5bc2-4294-a064-3f8d5c6877b9 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:38.112330 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b4465-7430-4aba-ac63-5595a745ced2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:38.112923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b4465-7430-4aba-ac63-5595a745ced2 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:38.113204 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b4465-7430-4aba-ac63-5595a745ced2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:38.113477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b4465-7430-4aba-ac63-5595a745ced2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:38.114191 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303b4465-7430-4aba-ac63-5595a745ced2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:39:38.114497 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303b4465-7430-4aba-ac63-5595a745ced2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:38.116585 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303b4465-7430-4aba-ac63-5595a745ced2 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:38.128002 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1247/2519] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:39:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:39:38.132713 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60609fe0-4d05-479a-a6bc-bc10c1397b10 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:38.141816 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60609fe0-4d05-479a-a6bc-bc10c1397b10 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:38.146453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd055b2-5f71-41f4-a829-85c3b0763435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:38.147078 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd055b2-5f71-41f4-a829-85c3b0763435 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:38.147398 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd055b2-5f71-41f4-a829-85c3b0763435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:38.147670 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd055b2-5f71-41f4-a829-85c3b0763435 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:38.148286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd055b2-5f71-41f4-a829-85c3b0763435 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:38.150240 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd055b2-5f71-41f4-a829-85c3b0763435 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:38.176629 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1273/2520] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:39:38 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:39:38.181531 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b4465-7430-4aba-ac63-5595a745ced2 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:38.191780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b4465-7430-4aba-ac63-5595a745ced2 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:38.196554 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:38.249991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25b859-366a-4f49-8726-8f5ee038c3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:38.250616 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25b859-366a-4f49-8726-8f5ee038c3d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:38.250893 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25b859-366a-4f49-8726-8f5ee038c3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:38.251162 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25b859-366a-4f49-8726-8f5ee038c3d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:38.251791 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25b859-366a-4f49-8726-8f5ee038c3d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:38.253704 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25b859-366a-4f49-8726-8f5ee038c3d6 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:38.274354 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1248/2521] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:39:38 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:39:38.278527 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd055b2-5f71-41f4-a829-85c3b0763435 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:38.289994 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd055b2-5f71-41f4-a829-85c3b0763435 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:38.294661 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b06680-f32a-47de-9924-f250619de7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:38.295258 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b06680-f32a-47de-9924-f250619de7b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:38.295539 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b06680-f32a-47de-9924-f250619de7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:38.295815 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b06680-f32a-47de-9924-f250619de7b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:38.296454 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b06680-f32a-47de-9924-f250619de7b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:38.298095 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b06680-f32a-47de-9924-f250619de7b8 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:38.308574 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1274/2522] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:39:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:39:38.312375 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25b859-366a-4f49-8726-8f5ee038c3d6 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:38.322661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25b859-366a-4f49-8726-8f5ee038c3d6 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:38.327173 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3755394e-dc80-4de3-a800-aebc284ef56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:38.327753 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3755394e-dc80-4de3-a800-aebc284ef56d admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:38.328045 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3755394e-dc80-4de3-a800-aebc284ef56d admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:38.328297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3755394e-dc80-4de3-a800-aebc284ef56d admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:38.328949 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3755394e-dc80-4de3-a800-aebc284ef56d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:38.330565 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3755394e-dc80-4de3-a800-aebc284ef56d admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:38.341518 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1249/2523] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:39:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:39:38.345500 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b06680-f32a-47de-9924-f250619de7b8 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:38.355534 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b06680-f32a-47de-9924-f250619de7b8 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:38.360383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00ae24-8ef0-419e-b200-a4f86fd99f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:38.361016 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00ae24-8ef0-419e-b200-a4f86fd99f5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:38.361290 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00ae24-8ef0-419e-b200-a4f86fd99f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:38.361561 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00ae24-8ef0-419e-b200-a4f86fd99f5a admin admin] PATH_INFO: `/v3/projects/804187f71b4042c5bcf86c6d3105f4f5/users/f8b5a1be6af848c5a3d318f3fb8f48b3/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:38.375767 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b00ae24-8ef0-419e-b200-a4f86fd99f5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=804187f71b4042c5bcf86c6d3105f4f5, user_id=f8b5a1be6af848c5a3d318f3fb8f48b3, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:38.378535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b00ae24-8ef0-419e-b200-a4f86fd99f5a admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:38.391956 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1275/2524] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:39:38 2022] PUT /identity/v3/projects/804187f71b4042c5bcf86c6d3105f4f5/users/f8b5a1be6af848c5a3d318f3fb8f48b3/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:39:38.402576 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f21b5-4e9b-47e7-8f65-08d333f2a365 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:38.402858 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f21b5-4e9b-47e7-8f65-08d333f2a365 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:38.403148 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f21b5-4e9b-47e7-8f65-08d333f2a365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:38.429926 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-f37f21b5-4e9b-47e7-8f65-08d333f2a365 None None] MFA Rules not processed for user `f8b5a1be6af848c5a3d318f3fb8f48b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:38.431157 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-f37f21b5-4e9b-47e7-8f65-08d333f2a365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:38.494520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1250/2525] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:38.505098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00ae24-8ef0-419e-b200-a4f86fd99f5a admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:38.507286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-3b00ae24-8ef0-419e-b200-a4f86fd99f5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:38.538016 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:38.565848 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00ae24-8ef0-419e-b200-a4f86fd99f5a admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:38.570087 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e556d0a-6c13-4a7d-8493-49c31a78024c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:38.570690 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e556d0a-6c13-4a7d-8493-49c31a78024c service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:38.570971 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e556d0a-6c13-4a7d-8493-49c31a78024c service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:38.571246 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e556d0a-6c13-4a7d-8493-49c31a78024c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:38.579477 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e556d0a-6c13-4a7d-8493-49c31a78024c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:38.580883 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e556d0a-6c13-4a7d-8493-49c31a78024c service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:38.589671 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1276/2526] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:39:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:39:40.127497 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f21b5-4e9b-47e7-8f65-08d333f2a365 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:40.128899 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-f37f21b5-4e9b-47e7-8f65-08d333f2a365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:40.198393 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f21b5-4e9b-47e7-8f65-08d333f2a365 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:40.203355 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4dd80-7944-4a91-bbcb-a14453279aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:40.204279 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4dd80-7944-4a91-bbcb-a14453279aba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:40.204654 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4dd80-7944-4a91-bbcb-a14453279aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:40.204931 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4dd80-7944-4a91-bbcb-a14453279aba admin admin] PATH_INFO: `/v3/users/b9bb121f42dc427384c1827ca4f6b58c` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:40.206289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4dd80-7944-4a91-bbcb-a14453279aba admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9bb121f42dc427384c1827ca4f6b58c)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:40.208480 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4dd80-7944-4a91-bbcb-a14453279aba admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:40.235263 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:40.263347 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-7fe4dd80-7944-4a91-bbcb-a14453279aba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9bb121f42dc427384c1827ca4f6b58c'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:40.264015 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-7fe4dd80-7944-4a91-bbcb-a14453279aba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9bb121f42dc427384c1827ca4f6b58c'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:40.264670 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-7fe4dd80-7944-4a91-bbcb-a14453279aba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9bb121f42dc427384c1827ca4f6b58c'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:40.271952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-7fe4dd80-7944-4a91-bbcb-a14453279aba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9bb121f42dc427384c1827ca4f6b58c'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:40.284871 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-7fe4dd80-7944-4a91-bbcb-a14453279aba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9bb121f42dc427384c1827ca4f6b58c'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:40.302599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1251/2527] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:39:40 2022] DELETE /identity/v3/users/b9bb121f42dc427384c1827ca4f6b58c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:40.306905 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83839b-4227-4f1f-bdf6-7004d77c4713 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:40.307982 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83839b-4227-4f1f-bdf6-7004d77c4713 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:40.308258 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83839b-4227-4f1f-bdf6-7004d77c4713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:40.330943 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-4d83839b-4227-4f1f-bdf6-7004d77c4713 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:40.332785 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-4d83839b-4227-4f1f-bdf6-7004d77c4713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:40.400484 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1277/2528] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:39:40.410116 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4dd80-7944-4a91-bbcb-a14453279aba admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:40.411502 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-7fe4dd80-7944-4a91-bbcb-a14453279aba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:40.468242 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4dd80-7944-4a91-bbcb-a14453279aba admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:40.472371 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13ba56-ab4e-4f10-98fc-9fe706787449 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:40.472968 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13ba56-ab4e-4f10-98fc-9fe706787449 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:40.473242 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13ba56-ab4e-4f10-98fc-9fe706787449 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:40.473565 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13ba56-ab4e-4f10-98fc-9fe706787449 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:40.488559 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13ba56-ab4e-4f10-98fc-9fe706787449 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:40.490076 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13ba56-ab4e-4f10-98fc-9fe706787449 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:40.500768 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1252/2529] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:39:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:39:40.769120 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83839b-4227-4f1f-bdf6-7004d77c4713 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:40.770667 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-4d83839b-4227-4f1f-bdf6-7004d77c4713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:40.786287 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83839b-4227-4f1f-bdf6-7004d77c4713 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:40.797039 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45545c1e-297a-47c6-9c9b-035e94326e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:40.797680 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45545c1e-297a-47c6-9c9b-035e94326e95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:40.797957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45545c1e-297a-47c6-9c9b-035e94326e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:40.798275 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45545c1e-297a-47c6-9c9b-035e94326e95 admin admin] PATH_INFO: `/v3/projects/eee3161e2ada49c5b144b913813e8bf6` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:40.799449 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45545c1e-297a-47c6-9c9b-035e94326e95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eee3161e2ada49c5b144b913813e8bf6)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:40.801351 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45545c1e-297a-47c6-9c9b-035e94326e95 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:40.852416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-45545c1e-297a-47c6-9c9b-035e94326e95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eee3161e2ada49c5b144b913813e8bf6', 'request_id': 'req-45545c1e-297a-47c6-9c9b-035e94326e95'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:40.858699 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-45545c1e-297a-47c6-9c9b-035e94326e95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eee3161e2ada49c5b144b913813e8bf6', 'request_id': 'req-45545c1e-297a-47c6-9c9b-035e94326e95'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:40.866848 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-45545c1e-297a-47c6-9c9b-035e94326e95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eee3161e2ada49c5b144b913813e8bf6', 'request_id': 'req-45545c1e-297a-47c6-9c9b-035e94326e95'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:40.876841 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-45545c1e-297a-47c6-9c9b-035e94326e95 admin admin] The token cache is being invalidate because project eee3161e2ada49c5b144b913813e8bf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:39:40.877122 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-45545c1e-297a-47c6-9c9b-035e94326e95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:40.877749 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-45545c1e-297a-47c6-9c9b-035e94326e95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:40.879202 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1278/2530] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:39:40 2022] DELETE /identity/v3/projects/eee3161e2ada49c5b144b913813e8bf6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:39:40.883760 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13ba56-ab4e-4f10-98fc-9fe706787449 service neutron] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:40.894344 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13ba56-ab4e-4f10-98fc-9fe706787449 service neutron] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:40.952089 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a94b46-18b4-4537-bccb-9353ba34bf92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:40.952711 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a94b46-18b4-4537-bccb-9353ba34bf92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:40.952989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a94b46-18b4-4537-bccb-9353ba34bf92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:40.953263 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a94b46-18b4-4537-bccb-9353ba34bf92 admin admin] PATH_INFO: `/v3/users/1d55bc257c1d4a16ae55f4844f961403` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:40.954404 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a94b46-18b4-4537-bccb-9353ba34bf92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d55bc257c1d4a16ae55f4844f961403)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:40.956322 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a94b46-18b4-4537-bccb-9353ba34bf92 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:40.996416 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:41.001855 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e4a94b46-18b4-4537-bccb-9353ba34bf92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d55bc257c1d4a16ae55f4844f961403'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:41.002782 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e4a94b46-18b4-4537-bccb-9353ba34bf92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d55bc257c1d4a16ae55f4844f961403'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:41.003743 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e4a94b46-18b4-4537-bccb-9353ba34bf92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d55bc257c1d4a16ae55f4844f961403'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:41.011703 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e4a94b46-18b4-4537-bccb-9353ba34bf92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d55bc257c1d4a16ae55f4844f961403'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:41.023473 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e4a94b46-18b4-4537-bccb-9353ba34bf92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d55bc257c1d4a16ae55f4844f961403'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:41.036334 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1253/2531] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:39:40 2022] DELETE /identity/v3/users/1d55bc257c1d4a16ae55f4844f961403 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:41.316613 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45545c1e-297a-47c6-9c9b-035e94326e95 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:41.318065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-45545c1e-297a-47c6-9c9b-035e94326e95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:41.331258 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45545c1e-297a-47c6-9c9b-035e94326e95 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:41.382125 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:41.393131 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9db831-82ef-4dee-9797-464198244726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:41.394171 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9db831-82ef-4dee-9797-464198244726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:41.394520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9db831-82ef-4dee-9797-464198244726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:41.395026 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9db831-82ef-4dee-9797-464198244726 admin admin] PATH_INFO: `/v3/projects/612f0044fcff4d8cb1af1627446a6834` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:41.396175 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9db831-82ef-4dee-9797-464198244726 admin admin] RBAC: Authorizing `identity:delete_project(project_id=612f0044fcff4d8cb1af1627446a6834)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:41.398106 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9db831-82ef-4dee-9797-464198244726 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:41.447184 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-bd9db831-82ef-4dee-9797-464198244726 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '612f0044fcff4d8cb1af1627446a6834', 'request_id': 'req-bd9db831-82ef-4dee-9797-464198244726'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:41.453525 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-bd9db831-82ef-4dee-9797-464198244726 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '612f0044fcff4d8cb1af1627446a6834', 'request_id': 'req-bd9db831-82ef-4dee-9797-464198244726'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:41.461817 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-bd9db831-82ef-4dee-9797-464198244726 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '612f0044fcff4d8cb1af1627446a6834', 'request_id': 'req-bd9db831-82ef-4dee-9797-464198244726'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:41.474270 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-bd9db831-82ef-4dee-9797-464198244726 admin admin] The token cache is being invalidate because project 612f0044fcff4d8cb1af1627446a6834 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:39:41.474560 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-bd9db831-82ef-4dee-9797-464198244726 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:41.475216 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-bd9db831-82ef-4dee-9797-464198244726 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:41.477255 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1279/2532] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:39:41 2022] DELETE /identity/v3/projects/612f0044fcff4d8cb1af1627446a6834 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:41.482367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9cdcc-3ad2-4d22-a66d-c784b0fb9dae None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:41.482837 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9cdcc-3ad2-4d22-a66d-c784b0fb9dae None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:41.482922 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9cdcc-3ad2-4d22-a66d-c784b0fb9dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:41.504054 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-14e9cdcc-3ad2-4d22-a66d-c784b0fb9dae None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:41.505263 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-14e9cdcc-3ad2-4d22-a66d-c784b0fb9dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:41.565802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1254/2533] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:41.573942 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f46e2c-892f-40f2-9463-0a5a748ce380 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:41.574927 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f46e2c-892f-40f2-9463-0a5a748ce380 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:41.574927 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f46e2c-892f-40f2-9463-0a5a748ce380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:41.601399 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-f5f46e2c-892f-40f2-9463-0a5a748ce380 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:41.602734 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-f5f46e2c-892f-40f2-9463-0a5a748ce380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:41.615236 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1280/2534] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:39:41.621066 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9cdcc-3ad2-4d22-a66d-c784b0fb9dae None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:41.631105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9cdcc-3ad2-4d22-a66d-c784b0fb9dae None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:41.636012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa7af35-1d85-413e-a9d2-df8be5b25361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:41.636607 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa7af35-1d85-413e-a9d2-df8be5b25361 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:41.636885 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa7af35-1d85-413e-a9d2-df8be5b25361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:41.637155 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa7af35-1d85-413e-a9d2-df8be5b25361 admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:41.637797 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa7af35-1d85-413e-a9d2-df8be5b25361 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:39:41.638128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa7af35-1d85-413e-a9d2-df8be5b25361 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:41.639726 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa7af35-1d85-413e-a9d2-df8be5b25361 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:41.652653 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1255/2535] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:39:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:39:41.656930 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f46e2c-892f-40f2-9463-0a5a748ce380 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:41.667415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f46e2c-892f-40f2-9463-0a5a748ce380 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:41.674130 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6d931-11d7-4de5-9554-4a47295bfc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:41.675045 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6d931-11d7-4de5-9554-4a47295bfc06 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:41.675649 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6d931-11d7-4de5-9554-4a47295bfc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:41.676008 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6d931-11d7-4de5-9554-4a47295bfc06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:41.677094 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc6d931-11d7-4de5-9554-4a47295bfc06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:41.679212 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc6d931-11d7-4de5-9554-4a47295bfc06 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:41.717344 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1281/2536] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:39:41 2022] POST /identity/v3/projects => generated 407 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:39:41.721254 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa7af35-1d85-413e-a9d2-df8be5b25361 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:41.746535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa7af35-1d85-413e-a9d2-df8be5b25361 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:41.834304 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15d40c-825f-4bc6-974f-9be825c34845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:41.835464 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15d40c-825f-4bc6-974f-9be825c34845 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:41.835746 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15d40c-825f-4bc6-974f-9be825c34845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:41.836013 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15d40c-825f-4bc6-974f-9be825c34845 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:41.836649 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15d40c-825f-4bc6-974f-9be825c34845 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:41.838598 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15d40c-825f-4bc6-974f-9be825c34845 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:41.861123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1256/2537] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:39:41 2022] POST /identity/v3/users => generated 304 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:39:41.865325 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6d931-11d7-4de5-9554-4a47295bfc06 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:41.875954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6d931-11d7-4de5-9554-4a47295bfc06 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:41.880757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a7d79-7aa6-435a-b486-ff3806098936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:41.881367 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a7d79-7aa6-435a-b486-ff3806098936 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:41.881645 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a7d79-7aa6-435a-b486-ff3806098936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:41.882505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a7d79-7aa6-435a-b486-ff3806098936 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:41.883141 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a7d79-7aa6-435a-b486-ff3806098936 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:41.884875 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a7d79-7aa6-435a-b486-ff3806098936 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:41.895651 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1282/2538] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:39:41.900189 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15d40c-825f-4bc6-974f-9be825c34845 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:41.910766 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15d40c-825f-4bc6-974f-9be825c34845 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:41.915471 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86d4d6-027d-4aad-b93a-26133e3f5593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:41.916064 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86d4d6-027d-4aad-b93a-26133e3f5593 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:41.916346 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86d4d6-027d-4aad-b93a-26133e3f5593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:41.916621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86d4d6-027d-4aad-b93a-26133e3f5593 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:41.917258 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d86d4d6-027d-4aad-b93a-26133e3f5593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:41.918912 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d86d4d6-027d-4aad-b93a-26133e3f5593 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:41.929456 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1257/2539] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:39:41.933380 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a7d79-7aa6-435a-b486-ff3806098936 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:41.943237 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a7d79-7aa6-435a-b486-ff3806098936 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:41.948454 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb5686-bc2c-4f78-81ce-160e3f54954a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:41.949246 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb5686-bc2c-4f78-81ce-160e3f54954a admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:41.949523 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb5686-bc2c-4f78-81ce-160e3f54954a admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:41.950176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb5686-bc2c-4f78-81ce-160e3f54954a admin admin] PATH_INFO: `/v3/projects/b5a68f37e3b5405ca4f799553043af76/users/88d14c5fe1e740908ca1d4114c959240/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:41.964992 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb5686-bc2c-4f78-81ce-160e3f54954a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5a68f37e3b5405ca4f799553043af76, user_id=88d14c5fe1e740908ca1d4114c959240, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:41.967638 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb5686-bc2c-4f78-81ce-160e3f54954a admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:41.979915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1283/2540] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:39:41 2022] PUT /identity/v3/projects/b5a68f37e3b5405ca4f799553043af76/users/88d14c5fe1e740908ca1d4114c959240/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:41.990453 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc1e79-7caf-4fee-93a0-6e7a3ccbb622 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:41.990708 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc1e79-7caf-4fee-93a0-6e7a3ccbb622 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:41.991066 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc1e79-7caf-4fee-93a0-6e7a3ccbb622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:42.018299 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-a0dc1e79-7caf-4fee-93a0-6e7a3ccbb622 None None] MFA Rules not processed for user `88d14c5fe1e740908ca1d4114c959240`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:42.019608 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-a0dc1e79-7caf-4fee-93a0-6e7a3ccbb622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:42.032117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:42.090628 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1258/2541] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:41 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:42.094979 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb5686-bc2c-4f78-81ce-160e3f54954a admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:42.158834 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb5686-bc2c-4f78-81ce-160e3f54954a admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:42.163743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f37dfc-6fb2-49e5-9fe9-ff23a7023826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:42.164351 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f37dfc-6fb2-49e5-9fe9-ff23a7023826 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:42.164626 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f37dfc-6fb2-49e5-9fe9-ff23a7023826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:42.164895 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f37dfc-6fb2-49e5-9fe9-ff23a7023826 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:42.165585 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f37dfc-6fb2-49e5-9fe9-ff23a7023826 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:42.167490 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f37dfc-6fb2-49e5-9fe9-ff23a7023826 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:42.193978 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1284/2542] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:39:42 2022] POST /identity/v3/projects => generated 409 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:39:42.198535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc1e79-7caf-4fee-93a0-6e7a3ccbb622 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:42.209065 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc1e79-7caf-4fee-93a0-6e7a3ccbb622 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:42.263727 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581038c-89a8-416e-bf96-7bec32e81b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:42.264323 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581038c-89a8-416e-bf96-7bec32e81b5c admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:42.264594 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581038c-89a8-416e-bf96-7bec32e81b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:42.264863 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581038c-89a8-416e-bf96-7bec32e81b5c admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:42.265483 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581038c-89a8-416e-bf96-7bec32e81b5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:42.267396 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581038c-89a8-416e-bf96-7bec32e81b5c admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:42.288251 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1259/2543] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:39:42 2022] POST /identity/v3/users => generated 311 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:39:42.292211 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f37dfc-6fb2-49e5-9fe9-ff23a7023826 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:42.298759 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:42.303091 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f37dfc-6fb2-49e5-9fe9-ff23a7023826 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:42.309508 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd13772-dbb6-4d4f-8c71-996fdde90588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:42.310358 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd13772-dbb6-4d4f-8c71-996fdde90588 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:42.310817 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd13772-dbb6-4d4f-8c71-996fdde90588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:42.311094 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd13772-dbb6-4d4f-8c71-996fdde90588 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:42.311737 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd13772-dbb6-4d4f-8c71-996fdde90588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:42.313313 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd13772-dbb6-4d4f-8c71-996fdde90588 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:42.324497 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1285/2544] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:39:42.328558 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581038c-89a8-416e-bf96-7bec32e81b5c admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:42.338560 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581038c-89a8-416e-bf96-7bec32e81b5c admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:42.343059 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b48184-ee3e-4eed-9548-0393c300f91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:42.343699 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b48184-ee3e-4eed-9548-0393c300f91f admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:42.343987 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b48184-ee3e-4eed-9548-0393c300f91f admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:42.344260 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b48184-ee3e-4eed-9548-0393c300f91f admin admin] PATH_INFO: `/v3/projects/01717972678a45b49006493a0f9e699b/users/ee1fdaf061644735b5fb816c9685934f/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:42.359207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b48184-ee3e-4eed-9548-0393c300f91f admin admin] RBAC: Authorizing `identity:create_grant(project_id=01717972678a45b49006493a0f9e699b, user_id=ee1fdaf061644735b5fb816c9685934f, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:42.361934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b48184-ee3e-4eed-9548-0393c300f91f admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:42.376156 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1260/2545] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:39:42 2022] PUT /identity/v3/projects/01717972678a45b49006493a0f9e699b/users/ee1fdaf061644735b5fb816c9685934f/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:42.380077 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd13772-dbb6-4d4f-8c71-996fdde90588 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:42.390178 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd13772-dbb6-4d4f-8c71-996fdde90588 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:42.446641 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f9db7-42a2-4833-989e-1cfd7d444fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:42.447236 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f9db7-42a2-4833-989e-1cfd7d444fc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:42.447511 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f9db7-42a2-4833-989e-1cfd7d444fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:42.447778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f9db7-42a2-4833-989e-1cfd7d444fc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:42.448397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827f9db7-42a2-4833-989e-1cfd7d444fc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:42.449964 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827f9db7-42a2-4833-989e-1cfd7d444fc9 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:42.459459 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1286/2546] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:39:42.464050 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b48184-ee3e-4eed-9548-0393c300f91f admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:42.474271 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b48184-ee3e-4eed-9548-0393c300f91f admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:42.479577 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50399c-bf73-4be8-9890-dc07684981c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:42.480220 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50399c-bf73-4be8-9890-dc07684981c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:42.480498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50399c-bf73-4be8-9890-dc07684981c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:42.481648 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50399c-bf73-4be8-9890-dc07684981c0 admin admin] PATH_INFO: `/v3/projects/01717972678a45b49006493a0f9e699b/users/ee1fdaf061644735b5fb816c9685934f/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:42.483618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a50399c-bf73-4be8-9890-dc07684981c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01717972678a45b49006493a0f9e699b, user_id=ee1fdaf061644735b5fb816c9685934f, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:42.486291 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a50399c-bf73-4be8-9890-dc07684981c0 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:42.496920 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1261/2547] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:39:42 2022] PUT /identity/v3/projects/01717972678a45b49006493a0f9e699b/users/ee1fdaf061644735b5fb816c9685934f/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:42.508134 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406cc625-877c-4132-a84c-4bc1e77268f7 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:42.508425 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406cc625-877c-4132-a84c-4bc1e77268f7 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:42.508704 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406cc625-877c-4132-a84c-4bc1e77268f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:42.536779 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-406cc625-877c-4132-a84c-4bc1e77268f7 None None] MFA Rules not processed for user `ee1fdaf061644735b5fb816c9685934f`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:42.538680 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-406cc625-877c-4132-a84c-4bc1e77268f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:42.605450 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1287/2548] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:42 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:42.618458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50399c-bf73-4be8-9890-dc07684981c0 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:42.620167 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-3a50399c-bf73-4be8-9890-dc07684981c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:42.681944 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50399c-bf73-4be8-9890-dc07684981c0 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:42.685680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add42c6-815b-4e42-9bd1-db7bb66a7da9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:42.686327 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add42c6-815b-4e42-9bd1-db7bb66a7da9 service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:42.686607 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add42c6-815b-4e42-9bd1-db7bb66a7da9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:42.686879 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add42c6-815b-4e42-9bd1-db7bb66a7da9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:42.696751 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0add42c6-815b-4e42-9bd1-db7bb66a7da9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:42.698208 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0add42c6-815b-4e42-9bd1-db7bb66a7da9 service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:42.708270 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1262/2549] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:39:42 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:39:42.861715 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406cc625-877c-4132-a84c-4bc1e77268f7 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:42.863113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-406cc625-877c-4132-a84c-4bc1e77268f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:42.874827 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406cc625-877c-4132-a84c-4bc1e77268f7 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:42.878473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e73e8-4ae2-41a6-a79e-a6d63b4c6e7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:42.879067 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e73e8-4ae2-41a6-a79e-a6d63b4c6e7a service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:42.879338 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e73e8-4ae2-41a6-a79e-a6d63b4c6e7a service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:42.879607 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e73e8-4ae2-41a6-a79e-a6d63b4c6e7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:42.943308 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e73e8-4ae2-41a6-a79e-a6d63b4c6e7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:42.945259 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e73e8-4ae2-41a6-a79e-a6d63b4c6e7a service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:42.954136 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1288/2550] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:39:42 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:39:52.703833 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=91,keystone:INSERT=5 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:52.949968 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=94 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:54.568569 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add42c6-815b-4e42-9bd1-db7bb66a7da9 service cinder] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:54.637507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add42c6-815b-4e42-9bd1-db7bb66a7da9 service cinder] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:54.642439 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8ca29-ff47-47b0-982b-1e770aeab54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:54.643123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8ca29-ff47-47b0-982b-1e770aeab54b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:54.643350 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8ca29-ff47-47b0-982b-1e770aeab54b admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:54.643623 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8ca29-ff47-47b0-982b-1e770aeab54b admin admin] PATH_INFO: `/v3/users/88d14c5fe1e740908ca1d4114c959240` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:54.644791 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8ca29-ff47-47b0-982b-1e770aeab54b admin admin] RBAC: Authorizing `identity:delete_user(user_id=88d14c5fe1e740908ca1d4114c959240)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:54.646927 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8ca29-ff47-47b0-982b-1e770aeab54b admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:54.692239 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-fbd8ca29-ff47-47b0-982b-1e770aeab54b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88d14c5fe1e740908ca1d4114c959240'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:54.692889 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-fbd8ca29-ff47-47b0-982b-1e770aeab54b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88d14c5fe1e740908ca1d4114c959240'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:54.693496 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-fbd8ca29-ff47-47b0-982b-1e770aeab54b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88d14c5fe1e740908ca1d4114c959240'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:54.700894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-fbd8ca29-ff47-47b0-982b-1e770aeab54b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88d14c5fe1e740908ca1d4114c959240'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:54.713810 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-fbd8ca29-ff47-47b0-982b-1e770aeab54b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88d14c5fe1e740908ca1d4114c959240'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:54.726675 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1263/2551] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:39:54 2022] DELETE /identity/v3/users/88d14c5fe1e740908ca1d4114c959240 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:54.731128 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48309622-cfc0-469e-82ca-4351409bdfd1 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:54.731853 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48309622-cfc0-469e-82ca-4351409bdfd1 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:54.731960 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48309622-cfc0-469e-82ca-4351409bdfd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:54.751435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-48309622-cfc0-469e-82ca-4351409bdfd1 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:54.752640 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-48309622-cfc0-469e-82ca-4351409bdfd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:54.824914 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1289/2552] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:54.834798 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8ca29-ff47-47b0-982b-1e770aeab54b admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:54.836781 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-fbd8ca29-ff47-47b0-982b-1e770aeab54b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:54.894064 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8ca29-ff47-47b0-982b-1e770aeab54b admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:54.897610 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32615b81-bf1d-4cc2-9adc-a70cb24f2a24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:54.898852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32615b81-bf1d-4cc2-9adc-a70cb24f2a24 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:54.899135 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32615b81-bf1d-4cc2-9adc-a70cb24f2a24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:54.899410 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32615b81-bf1d-4cc2-9adc-a70cb24f2a24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:54.908847 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32615b81-bf1d-4cc2-9adc-a70cb24f2a24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:54.910299 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32615b81-bf1d-4cc2-9adc-a70cb24f2a24 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:54.919703 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1264/2553] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:39:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:39:55.190579 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48309622-cfc0-469e-82ca-4351409bdfd1 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:55.191999 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-48309622-cfc0-469e-82ca-4351409bdfd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:55.207687 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48309622-cfc0-469e-82ca-4351409bdfd1 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:55.213630 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e697a7-33e0-4500-88fd-324a011bce2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:55.214296 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e697a7-33e0-4500-88fd-324a011bce2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:55.214578 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e697a7-33e0-4500-88fd-324a011bce2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:55.214856 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e697a7-33e0-4500-88fd-324a011bce2b admin admin] PATH_INFO: `/v3/projects/b5a68f37e3b5405ca4f799553043af76` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:55.215987 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e697a7-33e0-4500-88fd-324a011bce2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5a68f37e3b5405ca4f799553043af76)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:55.217871 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e697a7-33e0-4500-88fd-324a011bce2b admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:55.265709 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-80e697a7-33e0-4500-88fd-324a011bce2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5a68f37e3b5405ca4f799553043af76', 'request_id': 'req-80e697a7-33e0-4500-88fd-324a011bce2b'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:55.271945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-80e697a7-33e0-4500-88fd-324a011bce2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5a68f37e3b5405ca4f799553043af76', 'request_id': 'req-80e697a7-33e0-4500-88fd-324a011bce2b'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:55.279415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-80e697a7-33e0-4500-88fd-324a011bce2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5a68f37e3b5405ca4f799553043af76', 'request_id': 'req-80e697a7-33e0-4500-88fd-324a011bce2b'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:55.289713 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-80e697a7-33e0-4500-88fd-324a011bce2b admin admin] The token cache is being invalidate because project b5a68f37e3b5405ca4f799553043af76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:39:55.289995 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-80e697a7-33e0-4500-88fd-324a011bce2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:55.290630 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-80e697a7-33e0-4500-88fd-324a011bce2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:55.292093 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1290/2554] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:39:55 2022] DELETE /identity/v3/projects/b5a68f37e3b5405ca4f799553043af76 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:55.296004 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32615b81-bf1d-4cc2-9adc-a70cb24f2a24 service neutron] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:55.306665 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32615b81-bf1d-4cc2-9adc-a70cb24f2a24 service neutron] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:55.326939 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:55.366637 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c141a60b-7905-4b5b-afea-296690485482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:55.367839 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141a60b-7905-4b5b-afea-296690485482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:55.368120 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141a60b-7905-4b5b-afea-296690485482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:55.368392 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141a60b-7905-4b5b-afea-296690485482 admin admin] PATH_INFO: `/v3/users/ee1fdaf061644735b5fb816c9685934f` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:55.369518 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c141a60b-7905-4b5b-afea-296690485482 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee1fdaf061644735b5fb816c9685934f)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:55.371461 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c141a60b-7905-4b5b-afea-296690485482 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:55.417297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-c141a60b-7905-4b5b-afea-296690485482 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee1fdaf061644735b5fb816c9685934f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:55.417866 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-c141a60b-7905-4b5b-afea-296690485482 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee1fdaf061644735b5fb816c9685934f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:55.418505 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-c141a60b-7905-4b5b-afea-296690485482 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee1fdaf061644735b5fb816c9685934f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:55.425334 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-c141a60b-7905-4b5b-afea-296690485482 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee1fdaf061644735b5fb816c9685934f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:55.438537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-c141a60b-7905-4b5b-afea-296690485482 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee1fdaf061644735b5fb816c9685934f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:55.459716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1265/2555] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:39:55 2022] DELETE /identity/v3/users/ee1fdaf061644735b5fb816c9685934f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:55.727682 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e697a7-33e0-4500-88fd-324a011bce2b admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:55.729161 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-80e697a7-33e0-4500-88fd-324a011bce2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:55.743039 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e697a7-33e0-4500-88fd-324a011bce2b admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:55.810049 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13642fa-2a14-4ef0-9aa1-f9e3bf248da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:55.810682 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13642fa-2a14-4ef0-9aa1-f9e3bf248da2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:55.810982 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13642fa-2a14-4ef0-9aa1-f9e3bf248da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:55.811254 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13642fa-2a14-4ef0-9aa1-f9e3bf248da2 admin admin] PATH_INFO: `/v3/projects/01717972678a45b49006493a0f9e699b` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:55.812369 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13642fa-2a14-4ef0-9aa1-f9e3bf248da2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01717972678a45b49006493a0f9e699b)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:55.814434 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13642fa-2a14-4ef0-9aa1-f9e3bf248da2 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:55.844577 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:55.863322 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f13642fa-2a14-4ef0-9aa1-f9e3bf248da2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01717972678a45b49006493a0f9e699b', 'request_id': 'req-f13642fa-2a14-4ef0-9aa1-f9e3bf248da2'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:55.869409 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f13642fa-2a14-4ef0-9aa1-f9e3bf248da2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01717972678a45b49006493a0f9e699b', 'request_id': 'req-f13642fa-2a14-4ef0-9aa1-f9e3bf248da2'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:55.876666 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f13642fa-2a14-4ef0-9aa1-f9e3bf248da2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01717972678a45b49006493a0f9e699b', 'request_id': 'req-f13642fa-2a14-4ef0-9aa1-f9e3bf248da2'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:55.886957 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f13642fa-2a14-4ef0-9aa1-f9e3bf248da2 admin admin] The token cache is being invalidate because project 01717972678a45b49006493a0f9e699b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:39:55.887266 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f13642fa-2a14-4ef0-9aa1-f9e3bf248da2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:55.887889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f13642fa-2a14-4ef0-9aa1-f9e3bf248da2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:55.889633 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1291/2556] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:39:55 2022] DELETE /identity/v3/projects/01717972678a45b49006493a0f9e699b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:55.894444 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90bb70-d992-4119-aa2c-31cc80341e47 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:55.895001 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90bb70-d992-4119-aa2c-31cc80341e47 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:55.895001 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90bb70-d992-4119-aa2c-31cc80341e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:55.915455 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-7e90bb70-d992-4119-aa2c-31cc80341e47 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:55.916648 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-7e90bb70-d992-4119-aa2c-31cc80341e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:55.972991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1266/2557] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:55.981827 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48085f-5a8d-4b52-96f8-b88e47a3fb23 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:55.982140 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48085f-5a8d-4b52-96f8-b88e47a3fb23 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:55.982419 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48085f-5a8d-4b52-96f8-b88e47a3fb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:56.001716 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-8a48085f-5a8d-4b52-96f8-b88e47a3fb23 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:56.002966 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-8a48085f-5a8d-4b52-96f8-b88e47a3fb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:56.010816 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1292/2558] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:39:56.014889 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e90bb70-d992-4119-aa2c-31cc80341e47 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:56.024933 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e90bb70-d992-4119-aa2c-31cc80341e47 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:56.029643 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8eb523-c8d9-4b91-a2ee-fb1af4408db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:56.030280 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8eb523-c8d9-4b91-a2ee-fb1af4408db4 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:56.030580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8eb523-c8d9-4b91-a2ee-fb1af4408db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:56.030855 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8eb523-c8d9-4b91-a2ee-fb1af4408db4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:56.031500 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8eb523-c8d9-4b91-a2ee-fb1af4408db4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:39:56.031800 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8eb523-c8d9-4b91-a2ee-fb1af4408db4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:56.033394 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8eb523-c8d9-4b91-a2ee-fb1af4408db4 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:56.045452 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1267/2559] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:39:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:39:56.049504 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48085f-5a8d-4b52-96f8-b88e47a3fb23 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:56.058549 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48085f-5a8d-4b52-96f8-b88e47a3fb23 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:56.063125 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b213fa1-cc2f-4d88-b785-2993464e1ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:56.063709 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b213fa1-cc2f-4d88-b785-2993464e1ac4 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:56.063983 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b213fa1-cc2f-4d88-b785-2993464e1ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:56.064250 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b213fa1-cc2f-4d88-b785-2993464e1ac4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:56.064861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b213fa1-cc2f-4d88-b785-2993464e1ac4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:56.066754 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b213fa1-cc2f-4d88-b785-2993464e1ac4 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:56.095663 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1293/2560] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:39:56 2022] POST /identity/v3/projects => generated 359 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:39:56.100424 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8eb523-c8d9-4b91-a2ee-fb1af4408db4 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:56.113575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8eb523-c8d9-4b91-a2ee-fb1af4408db4 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:56.139847 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:56.172787 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de738dc0-9ad4-4f0d-9a86-edd8c98f07fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:56.173402 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de738dc0-9ad4-4f0d-9a86-edd8c98f07fe admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:56.173747 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de738dc0-9ad4-4f0d-9a86-edd8c98f07fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:56.174074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de738dc0-9ad4-4f0d-9a86-edd8c98f07fe admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:56.174732 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de738dc0-9ad4-4f0d-9a86-edd8c98f07fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:56.176674 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de738dc0-9ad4-4f0d-9a86-edd8c98f07fe admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:56.200721 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1268/2561] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:39:56 2022] POST /identity/v3/users => generated 280 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:39:56.202453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b213fa1-cc2f-4d88-b785-2993464e1ac4 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:56.213428 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b213fa1-cc2f-4d88-b785-2993464e1ac4 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:56.218121 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da559fba-72f7-4628-a085-f9c8ea6749b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:56.218717 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da559fba-72f7-4628-a085-f9c8ea6749b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:56.218997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da559fba-72f7-4628-a085-f9c8ea6749b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:56.219272 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da559fba-72f7-4628-a085-f9c8ea6749b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:56.219908 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da559fba-72f7-4628-a085-f9c8ea6749b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:56.221494 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da559fba-72f7-4628-a085-f9c8ea6749b6 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:56.231669 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1294/2562] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:39:56.235522 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de738dc0-9ad4-4f0d-9a86-edd8c98f07fe admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:56.245776 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de738dc0-9ad4-4f0d-9a86-edd8c98f07fe admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:56.250435 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e97ef5-ac8e-49cc-bad0-57a8d88dcd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:56.251044 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e97ef5-ac8e-49cc-bad0-57a8d88dcd76 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:56.251350 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e97ef5-ac8e-49cc-bad0-57a8d88dcd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:56.251622 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e97ef5-ac8e-49cc-bad0-57a8d88dcd76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:56.252253 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e97ef5-ac8e-49cc-bad0-57a8d88dcd76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:56.253828 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e97ef5-ac8e-49cc-bad0-57a8d88dcd76 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:56.264890 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1269/2563] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:39:56.269092 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da559fba-72f7-4628-a085-f9c8ea6749b6 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:56.279729 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da559fba-72f7-4628-a085-f9c8ea6749b6 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:56.284500 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ef45f-7676-4658-9a38-79d12edc5e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:56.285154 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ef45f-7676-4658-9a38-79d12edc5e6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:56.285434 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ef45f-7676-4658-9a38-79d12edc5e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:56.285705 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ef45f-7676-4658-9a38-79d12edc5e6b admin admin] PATH_INFO: `/v3/projects/deb1c82780a947a8a050f1b27a247469/users/ecc7a296ccf6485e94c607e51a518720/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:56.300158 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ef45f-7676-4658-9a38-79d12edc5e6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=deb1c82780a947a8a050f1b27a247469, user_id=ecc7a296ccf6485e94c607e51a518720, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:56.302803 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ef45f-7676-4658-9a38-79d12edc5e6b admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:56.315836 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1295/2564] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:39:56 2022] PUT /identity/v3/projects/deb1c82780a947a8a050f1b27a247469/users/ecc7a296ccf6485e94c607e51a518720/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:56.382053 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994d4f6-ea77-4cd9-b93b-46cef2173c77 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:56.382356 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994d4f6-ea77-4cd9-b93b-46cef2173c77 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:56.383138 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994d4f6-ea77-4cd9-b93b-46cef2173c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:56.411137 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-8994d4f6-ea77-4cd9-b93b-46cef2173c77 None None] MFA Rules not processed for user `ecc7a296ccf6485e94c607e51a518720`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:56.412351 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-8994d4f6-ea77-4cd9-b93b-46cef2173c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:56.480012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1270/2565] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:56 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:56.514697 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ef45f-7676-4658-9a38-79d12edc5e6b admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:56.516374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-9a4ef45f-7676-4658-9a38-79d12edc5e6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:56.574106 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ef45f-7676-4658-9a38-79d12edc5e6b admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:56.577682 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b8621-39a4-4f85-93cb-3b69d8645d40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:56.578312 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b8621-39a4-4f85-93cb-3b69d8645d40 service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:56.578588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b8621-39a4-4f85-93cb-3b69d8645d40 service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:56.578858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b8621-39a4-4f85-93cb-3b69d8645d40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:56.587292 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b8621-39a4-4f85-93cb-3b69d8645d40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:56.588696 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b8621-39a4-4f85-93cb-3b69d8645d40 service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:56.597508 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1296/2566] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:39:56 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:39:56.613482 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994d4f6-ea77-4cd9-b93b-46cef2173c77 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:56.676362 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994d4f6-ea77-4cd9-b93b-46cef2173c77 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:56.681411 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35876c69-f2bb-4f12-aac2-46f855f4003a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:56.682049 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35876c69-f2bb-4f12-aac2-46f855f4003a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:56.682343 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35876c69-f2bb-4f12-aac2-46f855f4003a admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:56.682618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35876c69-f2bb-4f12-aac2-46f855f4003a admin admin] PATH_INFO: `/v3/users/ecc7a296ccf6485e94c607e51a518720` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:56.683777 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35876c69-f2bb-4f12-aac2-46f855f4003a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecc7a296ccf6485e94c607e51a518720)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:56.685713 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35876c69-f2bb-4f12-aac2-46f855f4003a admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:56.729971 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:56.734816 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-35876c69-f2bb-4f12-aac2-46f855f4003a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecc7a296ccf6485e94c607e51a518720'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:56.735747 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-35876c69-f2bb-4f12-aac2-46f855f4003a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecc7a296ccf6485e94c607e51a518720'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:56.736796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-35876c69-f2bb-4f12-aac2-46f855f4003a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecc7a296ccf6485e94c607e51a518720'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:56.745834 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-35876c69-f2bb-4f12-aac2-46f855f4003a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecc7a296ccf6485e94c607e51a518720'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:56.758326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-35876c69-f2bb-4f12-aac2-46f855f4003a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecc7a296ccf6485e94c607e51a518720'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:56.777003 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1271/2567] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:39:56 2022] DELETE /identity/v3/users/ecc7a296ccf6485e94c607e51a518720 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:56.781446 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06a4c5-338f-40b3-a967-807801429031 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:56.781939 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06a4c5-338f-40b3-a967-807801429031 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:56.782073 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06a4c5-338f-40b3-a967-807801429031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:56.801661 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-dc06a4c5-338f-40b3-a967-807801429031 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:56.802915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-dc06a4c5-338f-40b3-a967-807801429031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:56.850709 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:56.864677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1297/2568] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:56.874880 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35876c69-f2bb-4f12-aac2-46f855f4003a admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:56.876285 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-35876c69-f2bb-4f12-aac2-46f855f4003a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:56.936419 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35876c69-f2bb-4f12-aac2-46f855f4003a admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:56.940170 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430fff58-c5d1-4d78-bced-a7e381c7744e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:56.940847 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fff58-c5d1-4d78-bced-a7e381c7744e service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:56.941126 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fff58-c5d1-4d78-bced-a7e381c7744e service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:56.941401 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fff58-c5d1-4d78-bced-a7e381c7744e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:56.950438 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430fff58-c5d1-4d78-bced-a7e381c7744e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:56.951849 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430fff58-c5d1-4d78-bced-a7e381c7744e service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:56.961205 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1272/2569] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:39:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:39:57.224567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06a4c5-338f-40b3-a967-807801429031 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:57.226050 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-dc06a4c5-338f-40b3-a967-807801429031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:57.239254 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06a4c5-338f-40b3-a967-807801429031 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:57.245731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282831a-e7a9-4a74-b2ce-6c19077298ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:57.246608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282831a-e7a9-4a74-b2ce-6c19077298ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:57.247003 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282831a-e7a9-4a74-b2ce-6c19077298ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:57.247280 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282831a-e7a9-4a74-b2ce-6c19077298ae admin admin] PATH_INFO: `/v3/projects/deb1c82780a947a8a050f1b27a247469` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:57.248411 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2282831a-e7a9-4a74-b2ce-6c19077298ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=deb1c82780a947a8a050f1b27a247469)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:57.250348 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2282831a-e7a9-4a74-b2ce-6c19077298ae admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:57.298453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-2282831a-e7a9-4a74-b2ce-6c19077298ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'deb1c82780a947a8a050f1b27a247469', 'request_id': 'req-2282831a-e7a9-4a74-b2ce-6c19077298ae'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:57.305511 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-2282831a-e7a9-4a74-b2ce-6c19077298ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'deb1c82780a947a8a050f1b27a247469', 'request_id': 'req-2282831a-e7a9-4a74-b2ce-6c19077298ae'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:57.312456 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-2282831a-e7a9-4a74-b2ce-6c19077298ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'deb1c82780a947a8a050f1b27a247469', 'request_id': 'req-2282831a-e7a9-4a74-b2ce-6c19077298ae'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:57.324299 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-2282831a-e7a9-4a74-b2ce-6c19077298ae admin admin] The token cache is being invalidate because project deb1c82780a947a8a050f1b27a247469 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:39:57.324571 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-2282831a-e7a9-4a74-b2ce-6c19077298ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:57.325466 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-2282831a-e7a9-4a74-b2ce-6c19077298ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:57.327002 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1298/2570] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:39:57 2022] DELETE /identity/v3/projects/deb1c82780a947a8a050f1b27a247469 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:57.332005 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660356b8-2acd-4f24-ba17-e65e284c28f2 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:57.332280 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660356b8-2acd-4f24-ba17-e65e284c28f2 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:57.332639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660356b8-2acd-4f24-ba17-e65e284c28f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:57.353923 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-660356b8-2acd-4f24-ba17-e65e284c28f2 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:57.355189 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-660356b8-2acd-4f24-ba17-e65e284c28f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:57.416652 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1273/2571] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:39:57.424621 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696c971-242b-45d7-b23e-c938b9aef968 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:57.424962 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696c971-242b-45d7-b23e-c938b9aef968 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:57.425268 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696c971-242b-45d7-b23e-c938b9aef968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:57.445258 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-7696c971-242b-45d7-b23e-c938b9aef968 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:57.446548 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-7696c971-242b-45d7-b23e-c938b9aef968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:57.457013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1299/2572] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:39:57.462126 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660356b8-2acd-4f24-ba17-e65e284c28f2 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:57.471665 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660356b8-2acd-4f24-ba17-e65e284c28f2 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:57.476145 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6a176-5dfe-4afc-99fd-da3b9918fd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:57.477307 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6a176-5dfe-4afc-99fd-da3b9918fd92 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:57.477579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6a176-5dfe-4afc-99fd-da3b9918fd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:57.477849 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6a176-5dfe-4afc-99fd-da3b9918fd92 admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:57.478518 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb6a176-5dfe-4afc-99fd-da3b9918fd92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:39:57.478818 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb6a176-5dfe-4afc-99fd-da3b9918fd92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:57.480453 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb6a176-5dfe-4afc-99fd-da3b9918fd92 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:57.492095 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1274/2573] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:39:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:39:57.496216 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7696c971-242b-45d7-b23e-c938b9aef968 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:57.505462 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7696c971-242b-45d7-b23e-c938b9aef968 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:57.510032 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f66f97-dd32-4aa5-adee-c9a7ef5510d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:57.510638 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f66f97-dd32-4aa5-adee-c9a7ef5510d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:57.510963 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f66f97-dd32-4aa5-adee-c9a7ef5510d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:57.511279 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f66f97-dd32-4aa5-adee-c9a7ef5510d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:57.512032 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f66f97-dd32-4aa5-adee-c9a7ef5510d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:57.513915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f66f97-dd32-4aa5-adee-c9a7ef5510d0 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:57.541543 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1300/2574] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:39:57 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:39:57.545818 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6a176-5dfe-4afc-99fd-da3b9918fd92 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:57.556065 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6a176-5dfe-4afc-99fd-da3b9918fd92 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:57.598934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:57.615841 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0b78b-7aa7-451c-86d4-88e5d961a672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:57.616448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0b78b-7aa7-451c-86d4-88e5d961a672 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:57.616727 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0b78b-7aa7-451c-86d4-88e5d961a672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:57.617027 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0b78b-7aa7-451c-86d4-88e5d961a672 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:57.617664 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0b78b-7aa7-451c-86d4-88e5d961a672 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:57.619606 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0b78b-7aa7-451c-86d4-88e5d961a672 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:57.640934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1275/2575] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:39:57 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:39:57.645221 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f66f97-dd32-4aa5-adee-c9a7ef5510d0 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:57.656000 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f66f97-dd32-4aa5-adee-c9a7ef5510d0 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:57.660576 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee98eb2-1dfc-49ae-8723-d14c13004910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:57.661163 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee98eb2-1dfc-49ae-8723-d14c13004910 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:57.661520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee98eb2-1dfc-49ae-8723-d14c13004910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:57.661794 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee98eb2-1dfc-49ae-8723-d14c13004910 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:57.662476 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee98eb2-1dfc-49ae-8723-d14c13004910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:57.664108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee98eb2-1dfc-49ae-8723-d14c13004910 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:57.673828 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1301/2576] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:39:57.677624 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0b78b-7aa7-451c-86d4-88e5d961a672 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:57.687563 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0b78b-7aa7-451c-86d4-88e5d961a672 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:57.692207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b70af-9176-44ac-89b7-be2d68918aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:57.692796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b70af-9176-44ac-89b7-be2d68918aee admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:57.693079 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b70af-9176-44ac-89b7-be2d68918aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:57.693354 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b70af-9176-44ac-89b7-be2d68918aee admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:57.693989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83b70af-9176-44ac-89b7-be2d68918aee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:57.696102 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83b70af-9176-44ac-89b7-be2d68918aee admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:57.706492 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1276/2577] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:39:57.710249 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee98eb2-1dfc-49ae-8723-d14c13004910 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:57.721099 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee98eb2-1dfc-49ae-8723-d14c13004910 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:57.725659 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea3ac95-122a-4781-92f6-5caee5824b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:57.726367 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3ac95-122a-4781-92f6-5caee5824b39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:57.726647 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3ac95-122a-4781-92f6-5caee5824b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:57.726918 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3ac95-122a-4781-92f6-5caee5824b39 admin admin] PATH_INFO: `/v3/projects/cfe6d038da334f89a92ef46bbbf6d5d3/users/6bd46cfb4c2c408780db90951d37c916/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:57.741443 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea3ac95-122a-4781-92f6-5caee5824b39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfe6d038da334f89a92ef46bbbf6d5d3, user_id=6bd46cfb4c2c408780db90951d37c916, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:57.744105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea3ac95-122a-4781-92f6-5caee5824b39 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:57.756951 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1302/2578] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:39:57 2022] PUT /identity/v3/projects/cfe6d038da334f89a92ef46bbbf6d5d3/users/6bd46cfb4c2c408780db90951d37c916/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:57.767592 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794eb224-32a7-42e2-904a-16638ff40f5f None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:57.768119 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794eb224-32a7-42e2-904a-16638ff40f5f None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:57.768240 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794eb224-32a7-42e2-904a-16638ff40f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:57.796870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea3ac95-122a-4781-92f6-5caee5824b39 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:57.797380 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-794eb224-32a7-42e2-904a-16638ff40f5f None None] MFA Rules not processed for user `6bd46cfb4c2c408780db90951d37c916`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:57.798207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-794eb224-32a7-42e2-904a-16638ff40f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:57.798875 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-9ea3ac95-122a-4781-92f6-5caee5824b39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:57.869374 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1277/2579] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:57.869726 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea3ac95-122a-4781-92f6-5caee5824b39 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:57.874525 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448b14c-1a51-4dc6-98d7-132e6549b678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:57.875182 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448b14c-1a51-4dc6-98d7-132e6549b678 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:57.875423 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448b14c-1a51-4dc6-98d7-132e6549b678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:57.875697 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448b14c-1a51-4dc6-98d7-132e6549b678 admin admin] PATH_INFO: `/v3/users/f8b5a1be6af848c5a3d318f3fb8f48b3` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:57.876842 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4448b14c-1a51-4dc6-98d7-132e6549b678 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8b5a1be6af848c5a3d318f3fb8f48b3)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:57.878776 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4448b14c-1a51-4dc6-98d7-132e6549b678 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:57.881427 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794eb224-32a7-42e2-904a-16638ff40f5f None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:57.882805 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-794eb224-32a7-42e2-904a-16638ff40f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:57.893645 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:57.934312 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-4448b14c-1a51-4dc6-98d7-132e6549b678 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8b5a1be6af848c5a3d318f3fb8f48b3'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:57.935191 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-4448b14c-1a51-4dc6-98d7-132e6549b678 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8b5a1be6af848c5a3d318f3fb8f48b3'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:57.944486 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-4448b14c-1a51-4dc6-98d7-132e6549b678 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8b5a1be6af848c5a3d318f3fb8f48b3'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:57.951453 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794eb224-32a7-42e2-904a-16638ff40f5f None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:57.955859 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef701a8-bb3a-4ce1-b3bb-4b0a87696bf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:57.956535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef701a8-bb3a-4ce1-b3bb-4b0a87696bf8 service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:57.956886 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef701a8-bb3a-4ce1-b3bb-4b0a87696bf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:57.957161 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef701a8-bb3a-4ce1-b3bb-4b0a87696bf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:57.959227 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-5ef701a8-bb3a-4ce1-b3bb-4b0a87696bf8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:57.960188 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-4448b14c-1a51-4dc6-98d7-132e6549b678 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8b5a1be6af848c5a3d318f3fb8f48b3'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:57.961726 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-4448b14c-1a51-4dc6-98d7-132e6549b678 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8b5a1be6af848c5a3d318f3fb8f48b3'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:57.971582 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef701a8-bb3a-4ce1-b3bb-4b0a87696bf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:57.973125 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef701a8-bb3a-4ce1-b3bb-4b0a87696bf8 service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:57.985051 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1303/2580] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:39:57 2022] DELETE /identity/v3/users/f8b5a1be6af848c5a3d318f3fb8f48b3 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:39:57.985257 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1278/2581] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:39:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:39:57.987584 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df80afb-831b-4b58-a6bd-0862047611cc None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:57.987943 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df80afb-831b-4b58-a6bd-0862047611cc None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:57.988334 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df80afb-831b-4b58-a6bd-0862047611cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:58.008011 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-5df80afb-831b-4b58-a6bd-0862047611cc None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:58.009301 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-5df80afb-831b-4b58-a6bd-0862047611cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:58.076223 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1304/2582] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:39:58.082989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef701a8-bb3a-4ce1-b3bb-4b0a87696bf8 service cinder] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:58.084479 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-5ef701a8-bb3a-4ce1-b3bb-4b0a87696bf8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:58.147115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef701a8-bb3a-4ce1-b3bb-4b0a87696bf8 service cinder] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:58.157495 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:58.205441 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178c577-f128-43d6-97e0-e27246eaef2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:58.206166 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178c577-f128-43d6-97e0-e27246eaef2e service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:58.206447 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178c577-f128-43d6-97e0-e27246eaef2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:58.206785 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178c577-f128-43d6-97e0-e27246eaef2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:58.218370 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7178c577-f128-43d6-97e0-e27246eaef2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:58.219837 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7178c577-f128-43d6-97e0-e27246eaef2e service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:58.246188 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1279/2583] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:39:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:39:58.526108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df80afb-831b-4b58-a6bd-0862047611cc None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:58.527892 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-5df80afb-831b-4b58-a6bd-0862047611cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:58.540374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df80afb-831b-4b58-a6bd-0862047611cc None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:58.545353 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e3452-6531-46a9-8a8d-28cc317c35cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:58.546006 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e3452-6531-46a9-8a8d-28cc317c35cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:58.546321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e3452-6531-46a9-8a8d-28cc317c35cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:58.546759 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e3452-6531-46a9-8a8d-28cc317c35cf admin admin] PATH_INFO: `/v3/projects/804187f71b4042c5bcf86c6d3105f4f5` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:58.547914 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5e3452-6531-46a9-8a8d-28cc317c35cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=804187f71b4042c5bcf86c6d3105f4f5)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:58.549827 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5e3452-6531-46a9-8a8d-28cc317c35cf admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:58.601119 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7b5e3452-6531-46a9-8a8d-28cc317c35cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '804187f71b4042c5bcf86c6d3105f4f5', 'request_id': 'req-7b5e3452-6531-46a9-8a8d-28cc317c35cf'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:58.608834 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7b5e3452-6531-46a9-8a8d-28cc317c35cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '804187f71b4042c5bcf86c6d3105f4f5', 'request_id': 'req-7b5e3452-6531-46a9-8a8d-28cc317c35cf'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:58.616667 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7b5e3452-6531-46a9-8a8d-28cc317c35cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '804187f71b4042c5bcf86c6d3105f4f5', 'request_id': 'req-7b5e3452-6531-46a9-8a8d-28cc317c35cf'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:58.627634 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7b5e3452-6531-46a9-8a8d-28cc317c35cf admin admin] The token cache is being invalidate because project 804187f71b4042c5bcf86c6d3105f4f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:39:58.627922 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7b5e3452-6531-46a9-8a8d-28cc317c35cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:58.628562 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7b5e3452-6531-46a9-8a8d-28cc317c35cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:39:58.630059 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1305/2584] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:39:58 2022] DELETE /identity/v3/projects/804187f71b4042c5bcf86c6d3105f4f5 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:58.640486 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbb2aa-8927-4d2c-845d-36b139009652 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:58.640771 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbb2aa-8927-4d2c-845d-36b139009652 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:58.641046 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbb2aa-8927-4d2c-845d-36b139009652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:58.661118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-e2fbb2aa-8927-4d2c-845d-36b139009652 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:58.662369 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-e2fbb2aa-8927-4d2c-845d-36b139009652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:58.721566 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1280/2585] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:58.729250 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb86e3-6b92-43a1-92a0-7f3804b5a4ec None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:58.729554 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb86e3-6b92-43a1-92a0-7f3804b5a4ec None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:58.729822 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb86e3-6b92-43a1-92a0-7f3804b5a4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:58.749895 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-04cb86e3-6b92-43a1-92a0-7f3804b5a4ec None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:58.751121 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-04cb86e3-6b92-43a1-92a0-7f3804b5a4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:58.759434 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1306/2586] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:58.763552 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbb2aa-8927-4d2c-845d-36b139009652 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:58.773527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbb2aa-8927-4d2c-845d-36b139009652 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:58.778099 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853480d-6442-4636-93e6-4bd16b7ce9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:58.778687 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853480d-6442-4636-93e6-4bd16b7ce9be admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:58.778963 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853480d-6442-4636-93e6-4bd16b7ce9be admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:58.779233 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853480d-6442-4636-93e6-4bd16b7ce9be admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:58.779869 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853480d-6442-4636-93e6-4bd16b7ce9be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:39:58.780167 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853480d-6442-4636-93e6-4bd16b7ce9be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:58.781749 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853480d-6442-4636-93e6-4bd16b7ce9be admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:58.793662 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1281/2587] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:39:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:39:58.797920 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cb86e3-6b92-43a1-92a0-7f3804b5a4ec None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:58.807891 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cb86e3-6b92-43a1-92a0-7f3804b5a4ec None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:58.812594 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2fa08-f27e-4cb0-aceb-c49c1e6a0fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:58.813137 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2fa08-f27e-4cb0-aceb-c49c1e6a0fef admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:58.813415 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2fa08-f27e-4cb0-aceb-c49c1e6a0fef admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:58.813685 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2fa08-f27e-4cb0-aceb-c49c1e6a0fef admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:58.814355 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c2fa08-f27e-4cb0-aceb-c49c1e6a0fef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:58.816236 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c2fa08-f27e-4cb0-aceb-c49c1e6a0fef admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:58.822295 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:58.846085 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1307/2588] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:39:58 2022] POST /identity/v3/projects => generated 403 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:39:58.850176 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853480d-6442-4636-93e6-4bd16b7ce9be admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:58.860631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853480d-6442-4636-93e6-4bd16b7ce9be admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:58.916528 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec75672-e874-4f78-8982-1b88e4e21652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:58.917274 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec75672-e874-4f78-8982-1b88e4e21652 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:58.917537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec75672-e874-4f78-8982-1b88e4e21652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:58.917818 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec75672-e874-4f78-8982-1b88e4e21652 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:58.918482 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec75672-e874-4f78-8982-1b88e4e21652 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:58.920398 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec75672-e874-4f78-8982-1b88e4e21652 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:58.941356 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1282/2589] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:39:58 2022] POST /identity/v3/users => generated 302 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:39:58.945370 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2fa08-f27e-4cb0-aceb-c49c1e6a0fef admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:58.955705 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2fa08-f27e-4cb0-aceb-c49c1e6a0fef admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:58.960271 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c632d5b6-500c-4494-be3c-2c0954d32aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:58.961404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632d5b6-500c-4494-be3c-2c0954d32aaa admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:58.961683 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632d5b6-500c-4494-be3c-2c0954d32aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:58.961955 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632d5b6-500c-4494-be3c-2c0954d32aaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:58.962622 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c632d5b6-500c-4494-be3c-2c0954d32aaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:58.964211 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c632d5b6-500c-4494-be3c-2c0954d32aaa admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:58.974042 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1308/2590] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:39:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:39:58.978392 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec75672-e874-4f78-8982-1b88e4e21652 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:58.988542 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec75672-e874-4f78-8982-1b88e4e21652 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:58.993545 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39741ace-2769-48e2-b8a6-87ef0d721591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:58.994158 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39741ace-2769-48e2-b8a6-87ef0d721591 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:58.994432 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39741ace-2769-48e2-b8a6-87ef0d721591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:58.994701 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39741ace-2769-48e2-b8a6-87ef0d721591 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:58.995326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39741ace-2769-48e2-b8a6-87ef0d721591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:58.996894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39741ace-2769-48e2-b8a6-87ef0d721591 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:59.007112 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1283/2591] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:39:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:39:59.011226 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c632d5b6-500c-4494-be3c-2c0954d32aaa admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:59.021444 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c632d5b6-500c-4494-be3c-2c0954d32aaa admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:59.026419 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c618671-4fbc-49c3-b07a-45634626ee16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:59.027078 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c618671-4fbc-49c3-b07a-45634626ee16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:59.027353 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c618671-4fbc-49c3-b07a-45634626ee16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:59.027623 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c618671-4fbc-49c3-b07a-45634626ee16 admin admin] PATH_INFO: `/v3/projects/3fcad9a945bc41c496612228154f7e64/users/0ec7ee95a77b4db6b266817ddc2782e8/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:59.042419 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c618671-4fbc-49c3-b07a-45634626ee16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fcad9a945bc41c496612228154f7e64, user_id=0ec7ee95a77b4db6b266817ddc2782e8, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:59.045029 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c618671-4fbc-49c3-b07a-45634626ee16 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:59.057727 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1309/2592] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:39:59 2022] PUT /identity/v3/projects/3fcad9a945bc41c496612228154f7e64/users/0ec7ee95a77b4db6b266817ddc2782e8/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:39:59.062972 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8f772-bae5-4058-81bb-e4e085243f35 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:59.063204 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8f772-bae5-4058-81bb-e4e085243f35 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:59.063510 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8f772-bae5-4058-81bb-e4e085243f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:59.082003 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-93a8f772-bae5-4058-81bb-e4e085243f35 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:59.083287 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-93a8f772-bae5-4058-81bb-e4e085243f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:59.100229 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:39:59.144053 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1284/2593] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:39:59.153737 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c618671-4fbc-49c3-b07a-45634626ee16 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:39:59.155437 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-8c618671-4fbc-49c3-b07a-45634626ee16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:39:59.211760 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c618671-4fbc-49c3-b07a-45634626ee16 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:39:59.215389 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804102b-93d1-479d-ac98-48a1544d3d67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:39:59.215976 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804102b-93d1-479d-ac98-48a1544d3d67 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:59.216250 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804102b-93d1-479d-ac98-48a1544d3d67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:59.216514 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804102b-93d1-479d-ac98-48a1544d3d67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:59.225784 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3804102b-93d1-479d-ac98-48a1544d3d67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:39:59.227733 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3804102b-93d1-479d-ac98-48a1544d3d67 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:39:59.237374 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1310/2594] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:39:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:39:59.926891 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16ac1e-86ae-4c17-97e9-5c9717390754 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:39:59.927177 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16ac1e-86ae-4c17-97e9-5c9717390754 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:39:59.927448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16ac1e-86ae-4c17-97e9-5c9717390754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:39:59.958974 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-6f16ac1e-86ae-4c17-97e9-5c9717390754 None None] MFA Rules not processed for user `0ec7ee95a77b4db6b266817ddc2782e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:39:59.960232 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-6f16ac1e-86ae-4c17-97e9-5c9717390754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:00.033758 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1285/2595] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:39:59 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:00.038284 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804102b-93d1-479d-ac98-48a1544d3d67 service neutron] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:00.049404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804102b-93d1-479d-ac98-48a1544d3d67 service neutron] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:00.055809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84848a51-d0e4-4eb7-bdd8-cabb284c1cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:00.056829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84848a51-d0e4-4eb7-bdd8-cabb284c1cbe admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:00.057332 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84848a51-d0e4-4eb7-bdd8-cabb284c1cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:00.058161 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84848a51-d0e4-4eb7-bdd8-cabb284c1cbe admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:00.058962 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84848a51-d0e4-4eb7-bdd8-cabb284c1cbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:00.060852 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84848a51-d0e4-4eb7-bdd8-cabb284c1cbe admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:00.092303 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1311/2596] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:40:00 2022] POST /identity/v3/projects => generated 401 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:40:00.093852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16ac1e-86ae-4c17-97e9-5c9717390754 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:00.105922 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16ac1e-86ae-4c17-97e9-5c9717390754 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:00.163120 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e593e1-6366-45c9-870a-1e9b9c873d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:00.164252 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e593e1-6366-45c9-870a-1e9b9c873d7f admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:00.164548 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e593e1-6366-45c9-870a-1e9b9c873d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:00.164823 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e593e1-6366-45c9-870a-1e9b9c873d7f admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:00.165476 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e593e1-6366-45c9-870a-1e9b9c873d7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:00.167421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e593e1-6366-45c9-870a-1e9b9c873d7f admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:00.189878 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1286/2597] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:40:00 2022] POST /identity/v3/users => generated 307 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:40:00.194930 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84848a51-d0e4-4eb7-bdd8-cabb284c1cbe admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:00.205636 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84848a51-d0e4-4eb7-bdd8-cabb284c1cbe admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:00.210393 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff857c-2d64-432a-add5-2f35922c2fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:00.210998 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff857c-2d64-432a-add5-2f35922c2fa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:00.211301 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff857c-2d64-432a-add5-2f35922c2fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:00.211574 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff857c-2d64-432a-add5-2f35922c2fa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:00.212724 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ff857c-2d64-432a-add5-2f35922c2fa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:00.215074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ff857c-2d64-432a-add5-2f35922c2fa7 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:00.227721 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1312/2598] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:40:00.232177 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e593e1-6366-45c9-870a-1e9b9c873d7f admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:00.242965 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e593e1-6366-45c9-870a-1e9b9c873d7f admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:00.247620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d2ce9-5150-4d99-916a-7c2bac1a17bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:00.248264 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d2ce9-5150-4d99-916a-7c2bac1a17bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:00.248546 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d2ce9-5150-4d99-916a-7c2bac1a17bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:00.248822 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d2ce9-5150-4d99-916a-7c2bac1a17bd admin admin] PATH_INFO: `/v3/projects/aced0a6960cc4ab1840147f0d6e0a1eb/users/bfffe07120354a29862ed7f9c86855e8/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:00.267398 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d2ce9-5150-4d99-916a-7c2bac1a17bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=aced0a6960cc4ab1840147f0d6e0a1eb, user_id=bfffe07120354a29862ed7f9c86855e8, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:00.270105 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d2ce9-5150-4d99-916a-7c2bac1a17bd admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:00.283463 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1287/2599] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:40:00 2022] PUT /identity/v3/projects/aced0a6960cc4ab1840147f0d6e0a1eb/users/bfffe07120354a29862ed7f9c86855e8/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:00.287434 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff857c-2d64-432a-add5-2f35922c2fa7 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:00.298005 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff857c-2d64-432a-add5-2f35922c2fa7 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:00.355914 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf57970-fa34-4f78-bce3-e0f2dd3b0e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:00.356841 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf57970-fa34-4f78-bce3-e0f2dd3b0e2e admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:00.357061 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf57970-fa34-4f78-bce3-e0f2dd3b0e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:00.357330 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf57970-fa34-4f78-bce3-e0f2dd3b0e2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:00.357986 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf57970-fa34-4f78-bce3-e0f2dd3b0e2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:00.359608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf57970-fa34-4f78-bce3-e0f2dd3b0e2e admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:00.370102 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1313/2600] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:40:00.374803 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d2ce9-5150-4d99-916a-7c2bac1a17bd admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:00.387187 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d2ce9-5150-4d99-916a-7c2bac1a17bd admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:00.392112 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ce4ea-5e57-40b2-acfd-66bd3cc889c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:00.392766 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ce4ea-5e57-40b2-acfd-66bd3cc889c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:00.393044 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ce4ea-5e57-40b2-acfd-66bd3cc889c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:00.393314 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ce4ea-5e57-40b2-acfd-66bd3cc889c9 admin admin] PATH_INFO: `/v3/projects/aced0a6960cc4ab1840147f0d6e0a1eb/users/bfffe07120354a29862ed7f9c86855e8/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:00.395278 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ce4ea-5e57-40b2-acfd-66bd3cc889c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aced0a6960cc4ab1840147f0d6e0a1eb, user_id=bfffe07120354a29862ed7f9c86855e8, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:00.397950 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ce4ea-5e57-40b2-acfd-66bd3cc889c9 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:00.409276 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1288/2601] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:40:00 2022] PUT /identity/v3/projects/aced0a6960cc4ab1840147f0d6e0a1eb/users/bfffe07120354a29862ed7f9c86855e8/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:01.050286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e4085-fa41-4044-a2e6-09e5bc926bc1 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:01.050794 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e4085-fa41-4044-a2e6-09e5bc926bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:01.050890 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e4085-fa41-4044-a2e6-09e5bc926bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:01.080022 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-480e4085-fa41-4044-a2e6-09e5bc926bc1 None None] MFA Rules not processed for user `bfffe07120354a29862ed7f9c86855e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:01.081245 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-480e4085-fa41-4044-a2e6-09e5bc926bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:01.086255 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:01.153596 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1314/2602] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:01 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:02.194286 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ce4ea-5e57-40b2-acfd-66bd3cc889c9 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:02.200352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:02.261759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ce4ea-5e57-40b2-acfd-66bd3cc889c9 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:02.266325 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f564d-59b0-4fa4-bbf2-5e442525d255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:02.266944 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f564d-59b0-4fa4-bbf2-5e442525d255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:02.267217 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f564d-59b0-4fa4-bbf2-5e442525d255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:02.267488 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f564d-59b0-4fa4-bbf2-5e442525d255 admin admin] PATH_INFO: `/v3/users/0ec7ee95a77b4db6b266817ddc2782e8` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:02.268579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1f564d-59b0-4fa4-bbf2-5e442525d255 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ec7ee95a77b4db6b266817ddc2782e8)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:02.270546 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1f564d-59b0-4fa4-bbf2-5e442525d255 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:02.315818 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-9e1f564d-59b0-4fa4-bbf2-5e442525d255 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ec7ee95a77b4db6b266817ddc2782e8'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:02.316539 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-9e1f564d-59b0-4fa4-bbf2-5e442525d255 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ec7ee95a77b4db6b266817ddc2782e8'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:02.317141 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-9e1f564d-59b0-4fa4-bbf2-5e442525d255 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ec7ee95a77b4db6b266817ddc2782e8'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:02.324554 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-9e1f564d-59b0-4fa4-bbf2-5e442525d255 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ec7ee95a77b4db6b266817ddc2782e8'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:02.336992 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-9e1f564d-59b0-4fa4-bbf2-5e442525d255 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ec7ee95a77b4db6b266817ddc2782e8'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:02.349168 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1289/2603] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:40:02 2022] DELETE /identity/v3/users/0ec7ee95a77b4db6b266817ddc2782e8 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:02.518425 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e4085-fa41-4044-a2e6-09e5bc926bc1 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:02.529396 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e4085-fa41-4044-a2e6-09e5bc926bc1 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:02.586668 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80989d-adbc-44e2-80ad-0c3ca86b6cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:02.587227 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80989d-adbc-44e2-80ad-0c3ca86b6cbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:02.587505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80989d-adbc-44e2-80ad-0c3ca86b6cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:02.587777 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80989d-adbc-44e2-80ad-0c3ca86b6cbc admin admin] PATH_INFO: `/v3/projects/3fcad9a945bc41c496612228154f7e64` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:02.588886 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae80989d-adbc-44e2-80ad-0c3ca86b6cbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fcad9a945bc41c496612228154f7e64)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:02.590857 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae80989d-adbc-44e2-80ad-0c3ca86b6cbc admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:02.637759 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-ae80989d-adbc-44e2-80ad-0c3ca86b6cbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fcad9a945bc41c496612228154f7e64', 'request_id': 'req-ae80989d-adbc-44e2-80ad-0c3ca86b6cbc'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:02.644059 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-ae80989d-adbc-44e2-80ad-0c3ca86b6cbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fcad9a945bc41c496612228154f7e64', 'request_id': 'req-ae80989d-adbc-44e2-80ad-0c3ca86b6cbc'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:02.653325 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-ae80989d-adbc-44e2-80ad-0c3ca86b6cbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fcad9a945bc41c496612228154f7e64', 'request_id': 'req-ae80989d-adbc-44e2-80ad-0c3ca86b6cbc'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:02.664267 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-ae80989d-adbc-44e2-80ad-0c3ca86b6cbc admin admin] The token cache is being invalidate because project 3fcad9a945bc41c496612228154f7e64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:40:02.664818 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-ae80989d-adbc-44e2-80ad-0c3ca86b6cbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:02.665618 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-ae80989d-adbc-44e2-80ad-0c3ca86b6cbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:02.667248 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1315/2604] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:40:02 2022] DELETE /identity/v3/projects/3fcad9a945bc41c496612228154f7e64 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:02.671417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f564d-59b0-4fa4-bbf2-5e442525d255 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:02.672838 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-9e1f564d-59b0-4fa4-bbf2-5e442525d255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:02.686675 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f564d-59b0-4fa4-bbf2-5e442525d255 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:02.745608 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785924b-a3a7-445b-95e1-ea8340be80d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:02.746288 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785924b-a3a7-445b-95e1-ea8340be80d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:02.746580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785924b-a3a7-445b-95e1-ea8340be80d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:02.746854 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785924b-a3a7-445b-95e1-ea8340be80d2 admin admin] PATH_INFO: `/v3/users/bfffe07120354a29862ed7f9c86855e8` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:02.747983 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785924b-a3a7-445b-95e1-ea8340be80d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfffe07120354a29862ed7f9c86855e8)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:02.750066 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785924b-a3a7-445b-95e1-ea8340be80d2 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:02.777825 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:02.813167 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-1785924b-a3a7-445b-95e1-ea8340be80d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfffe07120354a29862ed7f9c86855e8'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:02.814072 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-1785924b-a3a7-445b-95e1-ea8340be80d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfffe07120354a29862ed7f9c86855e8'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:02.814840 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-1785924b-a3a7-445b-95e1-ea8340be80d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfffe07120354a29862ed7f9c86855e8'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:02.823125 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-1785924b-a3a7-445b-95e1-ea8340be80d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfffe07120354a29862ed7f9c86855e8'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:02.836438 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-1785924b-a3a7-445b-95e1-ea8340be80d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfffe07120354a29862ed7f9c86855e8'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:02.849023 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1290/2605] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:40:02 2022] DELETE /identity/v3/users/bfffe07120354a29862ed7f9c86855e8 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:40:03.018882 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80989d-adbc-44e2-80ad-0c3ca86b6cbc admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:03.020522 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-ae80989d-adbc-44e2-80ad-0c3ca86b6cbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:03.033896 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80989d-adbc-44e2-80ad-0c3ca86b6cbc admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:03.065614 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:03.092947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d76b9-e1a0-42d1-880d-b937cc709fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:03.093572 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d76b9-e1a0-42d1-880d-b937cc709fb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:03.093849 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d76b9-e1a0-42d1-880d-b937cc709fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:03.094159 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d76b9-e1a0-42d1-880d-b937cc709fb1 admin admin] PATH_INFO: `/v3/projects/aced0a6960cc4ab1840147f0d6e0a1eb` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:03.095261 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d76b9-e1a0-42d1-880d-b937cc709fb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aced0a6960cc4ab1840147f0d6e0a1eb)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:03.097137 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d76b9-e1a0-42d1-880d-b937cc709fb1 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:03.144375 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-cf9d76b9-e1a0-42d1-880d-b937cc709fb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aced0a6960cc4ab1840147f0d6e0a1eb', 'request_id': 'req-cf9d76b9-e1a0-42d1-880d-b937cc709fb1'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:03.150460 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-cf9d76b9-e1a0-42d1-880d-b937cc709fb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aced0a6960cc4ab1840147f0d6e0a1eb', 'request_id': 'req-cf9d76b9-e1a0-42d1-880d-b937cc709fb1'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:03.157698 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-cf9d76b9-e1a0-42d1-880d-b937cc709fb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aced0a6960cc4ab1840147f0d6e0a1eb', 'request_id': 'req-cf9d76b9-e1a0-42d1-880d-b937cc709fb1'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:03.168837 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-cf9d76b9-e1a0-42d1-880d-b937cc709fb1 admin admin] The token cache is being invalidate because project aced0a6960cc4ab1840147f0d6e0a1eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:40:03.169128 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-cf9d76b9-e1a0-42d1-880d-b937cc709fb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:03.169754 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-cf9d76b9-e1a0-42d1-880d-b937cc709fb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:03.171218 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1316/2606] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:40:03 2022] DELETE /identity/v3/projects/aced0a6960cc4ab1840147f0d6e0a1eb => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:03.176140 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfa485-ea5a-482f-8c9a-967710e5b1c8 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:03.176442 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfa485-ea5a-482f-8c9a-967710e5b1c8 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:03.176723 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfa485-ea5a-482f-8c9a-967710e5b1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:03.196783 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-33dfa485-ea5a-482f-8c9a-967710e5b1c8 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:03.197991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-33dfa485-ea5a-482f-8c9a-967710e5b1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:03.258259 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1291/2607] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:03.265812 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18e945-04fd-45ad-bf62-2b5392376b3b None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:03.266131 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18e945-04fd-45ad-bf62-2b5392376b3b None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:03.266413 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18e945-04fd-45ad-bf62-2b5392376b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:03.285734 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-6b18e945-04fd-45ad-bf62-2b5392376b3b None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:03.286986 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-6b18e945-04fd-45ad-bf62-2b5392376b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:03.295652 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1317/2608] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:03.299684 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfa485-ea5a-482f-8c9a-967710e5b1c8 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:03.310429 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfa485-ea5a-482f-8c9a-967710e5b1c8 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:03.315091 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf4b00-02e3-4b9c-bd39-2560af1b53d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:03.315708 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf4b00-02e3-4b9c-bd39-2560af1b53d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:03.315982 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf4b00-02e3-4b9c-bd39-2560af1b53d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:03.316271 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf4b00-02e3-4b9c-bd39-2560af1b53d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:03.316905 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf4b00-02e3-4b9c-bd39-2560af1b53d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:40:03.317201 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf4b00-02e3-4b9c-bd39-2560af1b53d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:03.318838 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf4b00-02e3-4b9c-bd39-2560af1b53d4 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:03.330331 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1292/2609] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:40:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:40:03.334603 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18e945-04fd-45ad-bf62-2b5392376b3b None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:03.343982 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18e945-04fd-45ad-bf62-2b5392376b3b None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:03.348551 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873f33e-3da7-49a8-af13-6e7aa2ecf4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:03.349135 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873f33e-3da7-49a8-af13-6e7aa2ecf4b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:03.349411 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873f33e-3da7-49a8-af13-6e7aa2ecf4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:03.349683 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873f33e-3da7-49a8-af13-6e7aa2ecf4b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:03.350348 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7873f33e-3da7-49a8-af13-6e7aa2ecf4b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:03.352246 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7873f33e-3da7-49a8-af13-6e7aa2ecf4b3 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:03.381411 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1318/2610] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:40:03 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:40:03.385710 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf4b00-02e3-4b9c-bd39-2560af1b53d4 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:03.395837 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf4b00-02e3-4b9c-bd39-2560af1b53d4 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:03.453599 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a2d90-35d6-4e40-aaf1-35d6659f6fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:03.454224 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a2d90-35d6-4e40-aaf1-35d6659f6fb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:03.454506 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a2d90-35d6-4e40-aaf1-35d6659f6fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:03.454780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a2d90-35d6-4e40-aaf1-35d6659f6fb5 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:03.455401 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a2d90-35d6-4e40-aaf1-35d6659f6fb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:03.457319 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a2d90-35d6-4e40-aaf1-35d6659f6fb5 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:03.483197 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1293/2611] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:40:03 2022] POST /identity/v3/users => generated 282 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:40:03.487510 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873f33e-3da7-49a8-af13-6e7aa2ecf4b3 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:03.498452 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873f33e-3da7-49a8-af13-6e7aa2ecf4b3 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:03.503045 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51289c88-c05e-4557-bff1-edce531adc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:03.503637 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51289c88-c05e-4557-bff1-edce531adc5f admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:03.503915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51289c88-c05e-4557-bff1-edce531adc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:03.504187 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51289c88-c05e-4557-bff1-edce531adc5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:03.504846 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51289c88-c05e-4557-bff1-edce531adc5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:03.506467 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51289c88-c05e-4557-bff1-edce531adc5f admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:03.516255 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1319/2612] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:40:03.520119 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a2d90-35d6-4e40-aaf1-35d6659f6fb5 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:03.530580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a2d90-35d6-4e40-aaf1-35d6659f6fb5 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:03.535414 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb6f07-1f99-4ffc-9297-bedfa210a3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:03.536033 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb6f07-1f99-4ffc-9297-bedfa210a3e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:03.536307 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb6f07-1f99-4ffc-9297-bedfa210a3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:03.536579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb6f07-1f99-4ffc-9297-bedfa210a3e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:03.537240 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb6f07-1f99-4ffc-9297-bedfa210a3e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:03.538847 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb6f07-1f99-4ffc-9297-bedfa210a3e9 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:03.550293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1294/2613] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:40:03.553885 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51289c88-c05e-4557-bff1-edce531adc5f admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:03.564682 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51289c88-c05e-4557-bff1-edce531adc5f admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:03.569275 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71172676-e6f1-4fb8-8832-74d7b326d48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:03.569907 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71172676-e6f1-4fb8-8832-74d7b326d48c admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:03.570215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71172676-e6f1-4fb8-8832-74d7b326d48c admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:03.570488 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71172676-e6f1-4fb8-8832-74d7b326d48c admin admin] PATH_INFO: `/v3/projects/06571955c5b64e35aa6b95c5a45d7eb1/users/caf200f89fef4adc823a01e9f0f16c3e/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:03.584946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71172676-e6f1-4fb8-8832-74d7b326d48c admin admin] RBAC: Authorizing `identity:create_grant(project_id=06571955c5b64e35aa6b95c5a45d7eb1, user_id=caf200f89fef4adc823a01e9f0f16c3e, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:03.587685 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71172676-e6f1-4fb8-8832-74d7b326d48c admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:03.600748 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1320/2614] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:40:03 2022] PUT /identity/v3/projects/06571955c5b64e35aa6b95c5a45d7eb1/users/caf200f89fef4adc823a01e9f0f16c3e/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:03.611821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c251e8b-702d-464e-84dd-618796eefd04 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:03.612106 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c251e8b-702d-464e-84dd-618796eefd04 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:03.612382 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c251e8b-702d-464e-84dd-618796eefd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:03.631883 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:03.642484 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-4c251e8b-702d-464e-84dd-618796eefd04 None None] MFA Rules not processed for user `caf200f89fef4adc823a01e9f0f16c3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:03.643732 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-4c251e8b-702d-464e-84dd-618796eefd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:03.716835 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1295/2615] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:03.718086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71172676-e6f1-4fb8-8832-74d7b326d48c admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:03.813984 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71172676-e6f1-4fb8-8832-74d7b326d48c admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:03.819434 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11b659-6cf1-4202-bf8a-474212a1e5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:03.820122 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11b659-6cf1-4202-bf8a-474212a1e5b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:03.820548 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11b659-6cf1-4202-bf8a-474212a1e5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:03.820905 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11b659-6cf1-4202-bf8a-474212a1e5b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:03.821999 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a11b659-6cf1-4202-bf8a-474212a1e5b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:03.824288 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a11b659-6cf1-4202-bf8a-474212a1e5b4 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:03.829760 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:03.860812 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1321/2616] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:40:03 2022] POST /identity/v3/projects => generated 365 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:40:03.865283 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c251e8b-702d-464e-84dd-618796eefd04 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:03.889464 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c251e8b-702d-464e-84dd-618796eefd04 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:03.986014 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccbc416-c9c1-4881-9c2f-f487553201f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:03.986656 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbc416-c9c1-4881-9c2f-f487553201f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:03.986929 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbc416-c9c1-4881-9c2f-f487553201f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:03.987197 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbc416-c9c1-4881-9c2f-f487553201f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:03.987824 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccbc416-c9c1-4881-9c2f-f487553201f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:03.989724 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccbc416-c9c1-4881-9c2f-f487553201f8 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:04.011725 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1296/2617] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:40:03 2022] POST /identity/v3/users => generated 289 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:40:04.016080 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11b659-6cf1-4202-bf8a-474212a1e5b4 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:04.026763 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11b659-6cf1-4202-bf8a-474212a1e5b4 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:04.031451 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b3577-4419-4bdd-8764-b2c17589864f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:04.032049 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b3577-4419-4bdd-8764-b2c17589864f admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:04.032328 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b3577-4419-4bdd-8764-b2c17589864f admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:04.032603 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b3577-4419-4bdd-8764-b2c17589864f admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:04.033267 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9b3577-4419-4bdd-8764-b2c17589864f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:04.034884 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9b3577-4419-4bdd-8764-b2c17589864f admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:04.045893 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1322/2618] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:40:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:40:04.050045 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccbc416-c9c1-4881-9c2f-f487553201f8 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:04.061270 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccbc416-c9c1-4881-9c2f-f487553201f8 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:04.066228 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee490b-e8ac-4eaf-8e83-39612657c5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:04.066874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee490b-e8ac-4eaf-8e83-39612657c5cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:04.067148 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee490b-e8ac-4eaf-8e83-39612657c5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:04.067416 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee490b-e8ac-4eaf-8e83-39612657c5cd admin admin] PATH_INFO: `/v3/projects/d4f3cf0dcaf24043b7427f69fafa972c/users/0edf7eed720a4eb196bcc0abcd553409/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:04.081934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee490b-e8ac-4eaf-8e83-39612657c5cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4f3cf0dcaf24043b7427f69fafa972c, user_id=0edf7eed720a4eb196bcc0abcd553409, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:04.084596 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee490b-e8ac-4eaf-8e83-39612657c5cd admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:04.097379 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1297/2619] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:40:04 2022] PUT /identity/v3/projects/d4f3cf0dcaf24043b7427f69fafa972c/users/0edf7eed720a4eb196bcc0abcd553409/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:04.101951 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b3577-4419-4bdd-8764-b2c17589864f admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:04.112832 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b3577-4419-4bdd-8764-b2c17589864f admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:04.168801 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a159491-b962-44ef-8f5f-514a942ee506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:04.169325 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a159491-b962-44ef-8f5f-514a942ee506 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:04.169599 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a159491-b962-44ef-8f5f-514a942ee506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:04.169867 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a159491-b962-44ef-8f5f-514a942ee506 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:04.170590 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a159491-b962-44ef-8f5f-514a942ee506 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:04.172162 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a159491-b962-44ef-8f5f-514a942ee506 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:04.182260 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1323/2620] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:40:04 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:40:04.186527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee490b-e8ac-4eaf-8e83-39612657c5cd admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:04.201555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee490b-e8ac-4eaf-8e83-39612657c5cd admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:04.207364 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d911dc-7af6-4845-9e08-9df5124b9b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:04.208283 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d911dc-7af6-4845-9e08-9df5124b9b69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:04.208563 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d911dc-7af6-4845-9e08-9df5124b9b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:04.208834 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d911dc-7af6-4845-9e08-9df5124b9b69 admin admin] PATH_INFO: `/v3/projects/d4f3cf0dcaf24043b7427f69fafa972c/users/0edf7eed720a4eb196bcc0abcd553409/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:04.210821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d911dc-7af6-4845-9e08-9df5124b9b69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4f3cf0dcaf24043b7427f69fafa972c, user_id=0edf7eed720a4eb196bcc0abcd553409, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:04.213431 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d911dc-7af6-4845-9e08-9df5124b9b69 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:04.223752 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1298/2621] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:40:04 2022] PUT /identity/v3/projects/d4f3cf0dcaf24043b7427f69fafa972c/users/0edf7eed720a4eb196bcc0abcd553409/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:04.235904 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56de7a3-e9fc-4d88-8245-14b6e1dcc169 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:04.236190 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56de7a3-e9fc-4d88-8245-14b6e1dcc169 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:04.236498 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56de7a3-e9fc-4d88-8245-14b6e1dcc169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:04.264293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-d56de7a3-e9fc-4d88-8245-14b6e1dcc169 None None] MFA Rules not processed for user `0edf7eed720a4eb196bcc0abcd553409`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:04.265496 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-d56de7a3-e9fc-4d88-8245-14b6e1dcc169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:04.331614 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1324/2622] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:04 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:04.343333 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d911dc-7af6-4845-9e08-9df5124b9b69 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:04.345010 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-47d911dc-7af6-4845-9e08-9df5124b9b69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:04.403241 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d911dc-7af6-4845-9e08-9df5124b9b69 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:04.406811 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b9841c-7c59-477c-8734-5d5fd1fefece service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:04.407415 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9841c-7c59-477c-8734-5d5fd1fefece service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:04.407690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9841c-7c59-477c-8734-5d5fd1fefece service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:04.407958 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9841c-7c59-477c-8734-5d5fd1fefece service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:04.413783 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:04.419180 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b9841c-7c59-477c-8734-5d5fd1fefece service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:04.421036 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b9841c-7c59-477c-8734-5d5fd1fefece service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:04.431033 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1299/2623] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:40:04 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:40:04.842682 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56de7a3-e9fc-4d88-8245-14b6e1dcc169 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:04.899259 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:04.907185 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56de7a3-e9fc-4d88-8245-14b6e1dcc169 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:04.911810 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d1405-7cbd-4228-ade8-e9cdd7ac110d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:04.912426 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d1405-7cbd-4228-ade8-e9cdd7ac110d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:04.912701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d1405-7cbd-4228-ade8-e9cdd7ac110d admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:04.912968 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d1405-7cbd-4228-ade8-e9cdd7ac110d admin admin] PATH_INFO: `/v3/users/caf200f89fef4adc823a01e9f0f16c3e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:04.914125 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51d1405-7cbd-4228-ade8-e9cdd7ac110d admin admin] RBAC: Authorizing `identity:delete_user(user_id=caf200f89fef4adc823a01e9f0f16c3e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:04.916550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51d1405-7cbd-4228-ade8-e9cdd7ac110d admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:04.974275 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d51d1405-7cbd-4228-ade8-e9cdd7ac110d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caf200f89fef4adc823a01e9f0f16c3e'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:04.974953 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d51d1405-7cbd-4228-ade8-e9cdd7ac110d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caf200f89fef4adc823a01e9f0f16c3e'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:04.982644 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d51d1405-7cbd-4228-ade8-e9cdd7ac110d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caf200f89fef4adc823a01e9f0f16c3e'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:04.995014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d51d1405-7cbd-4228-ade8-e9cdd7ac110d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caf200f89fef4adc823a01e9f0f16c3e'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:04.995703 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d51d1405-7cbd-4228-ade8-e9cdd7ac110d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caf200f89fef4adc823a01e9f0f16c3e'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:05.009544 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1325/2624] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:40:04 2022] DELETE /identity/v3/users/caf200f89fef4adc823a01e9f0f16c3e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:05.013875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328e84d-c6ae-4eac-a138-0b4c1f580ed6 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:05.014200 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328e84d-c6ae-4eac-a138-0b4c1f580ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:05.014490 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328e84d-c6ae-4eac-a138-0b4c1f580ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:05.035431 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-0328e84d-c6ae-4eac-a138-0b4c1f580ed6 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:05.036650 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-0328e84d-c6ae-4eac-a138-0b4c1f580ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:05.094647 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1300/2625] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:05.104282 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d1405-7cbd-4228-ade8-e9cdd7ac110d admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:05.105732 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-d51d1405-7cbd-4228-ade8-e9cdd7ac110d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:05.162891 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d1405-7cbd-4228-ade8-e9cdd7ac110d admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:05.166524 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a40d53-a475-44f4-b5e3-fb088d7eedb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:05.167117 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a40d53-a475-44f4-b5e3-fb088d7eedb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:05.167391 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a40d53-a475-44f4-b5e3-fb088d7eedb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:05.167660 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a40d53-a475-44f4-b5e3-fb088d7eedb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:05.176417 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a40d53-a475-44f4-b5e3-fb088d7eedb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:05.177813 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a40d53-a475-44f4-b5e3-fb088d7eedb8 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:05.187952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1326/2626] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:40:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:40:05.454038 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0328e84d-c6ae-4eac-a138-0b4c1f580ed6 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:05.455738 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-0328e84d-c6ae-4eac-a138-0b4c1f580ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:05.472085 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0328e84d-c6ae-4eac-a138-0b4c1f580ed6 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:05.476998 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3af60e-b089-4ec7-a4df-135aa8057adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:05.478180 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3af60e-b089-4ec7-a4df-135aa8057adb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:05.478461 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3af60e-b089-4ec7-a4df-135aa8057adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:05.478734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3af60e-b089-4ec7-a4df-135aa8057adb admin admin] PATH_INFO: `/v3/projects/06571955c5b64e35aa6b95c5a45d7eb1` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:05.479966 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3af60e-b089-4ec7-a4df-135aa8057adb admin admin] RBAC: Authorizing `identity:delete_project(project_id=06571955c5b64e35aa6b95c5a45d7eb1)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:05.482446 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3af60e-b089-4ec7-a4df-135aa8057adb admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:05.529685 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6a3af60e-b089-4ec7-a4df-135aa8057adb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06571955c5b64e35aa6b95c5a45d7eb1', 'request_id': 'req-6a3af60e-b089-4ec7-a4df-135aa8057adb'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:05.535725 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6a3af60e-b089-4ec7-a4df-135aa8057adb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06571955c5b64e35aa6b95c5a45d7eb1', 'request_id': 'req-6a3af60e-b089-4ec7-a4df-135aa8057adb'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:05.541697 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6a3af60e-b089-4ec7-a4df-135aa8057adb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06571955c5b64e35aa6b95c5a45d7eb1', 'request_id': 'req-6a3af60e-b089-4ec7-a4df-135aa8057adb'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:05.553496 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6a3af60e-b089-4ec7-a4df-135aa8057adb admin admin] The token cache is being invalidate because project 06571955c5b64e35aa6b95c5a45d7eb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:40:05.553903 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6a3af60e-b089-4ec7-a4df-135aa8057adb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:05.554604 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6a3af60e-b089-4ec7-a4df-135aa8057adb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:05.556752 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1301/2627] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:40:05 2022] DELETE /identity/v3/projects/06571955c5b64e35aa6b95c5a45d7eb1 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:05.560772 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a40d53-a475-44f4-b5e3-fb088d7eedb8 service neutron] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:05.570479 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a40d53-a475-44f4-b5e3-fb088d7eedb8 service neutron] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:05.626973 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffea05a-904e-4709-b482-486f3e1a6947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:05.627600 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffea05a-904e-4709-b482-486f3e1a6947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:05.627882 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffea05a-904e-4709-b482-486f3e1a6947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:05.628163 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffea05a-904e-4709-b482-486f3e1a6947 admin admin] PATH_INFO: `/v3/users/0edf7eed720a4eb196bcc0abcd553409` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:05.629281 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffea05a-904e-4709-b482-486f3e1a6947 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0edf7eed720a4eb196bcc0abcd553409)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:05.631215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffea05a-904e-4709-b482-486f3e1a6947 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:05.651468 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:05.677726 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-0ffea05a-904e-4709-b482-486f3e1a6947 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0edf7eed720a4eb196bcc0abcd553409'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:05.678371 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-0ffea05a-904e-4709-b482-486f3e1a6947 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0edf7eed720a4eb196bcc0abcd553409'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:05.685825 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-0ffea05a-904e-4709-b482-486f3e1a6947 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0edf7eed720a4eb196bcc0abcd553409'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:05.697526 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-0ffea05a-904e-4709-b482-486f3e1a6947 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0edf7eed720a4eb196bcc0abcd553409'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:05.698116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-0ffea05a-904e-4709-b482-486f3e1a6947 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0edf7eed720a4eb196bcc0abcd553409'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:05.710786 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1327/2628] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:40:05 2022] DELETE /identity/v3/users/0edf7eed720a4eb196bcc0abcd553409 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:05.983846 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3af60e-b089-4ec7-a4df-135aa8057adb admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:05.985452 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-6a3af60e-b089-4ec7-a4df-135aa8057adb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:05.998778 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3af60e-b089-4ec7-a4df-135aa8057adb admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:06.057795 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9fce1-ecd2-4dcb-b5cd-760aac36ff29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:06.058526 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9fce1-ecd2-4dcb-b5cd-760aac36ff29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:06.059418 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9fce1-ecd2-4dcb-b5cd-760aac36ff29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:06.059718 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9fce1-ecd2-4dcb-b5cd-760aac36ff29 admin admin] PATH_INFO: `/v3/projects/d4f3cf0dcaf24043b7427f69fafa972c` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:06.060834 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c9fce1-ecd2-4dcb-b5cd-760aac36ff29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4f3cf0dcaf24043b7427f69fafa972c)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:06.062952 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c9fce1-ecd2-4dcb-b5cd-760aac36ff29 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:06.084834 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:06.113029 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-37c9fce1-ecd2-4dcb-b5cd-760aac36ff29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4f3cf0dcaf24043b7427f69fafa972c', 'request_id': 'req-37c9fce1-ecd2-4dcb-b5cd-760aac36ff29'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:06.119287 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-37c9fce1-ecd2-4dcb-b5cd-760aac36ff29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4f3cf0dcaf24043b7427f69fafa972c', 'request_id': 'req-37c9fce1-ecd2-4dcb-b5cd-760aac36ff29'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:06.124859 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-37c9fce1-ecd2-4dcb-b5cd-760aac36ff29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4f3cf0dcaf24043b7427f69fafa972c', 'request_id': 'req-37c9fce1-ecd2-4dcb-b5cd-760aac36ff29'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:06.136038 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-37c9fce1-ecd2-4dcb-b5cd-760aac36ff29 admin admin] The token cache is being invalidate because project d4f3cf0dcaf24043b7427f69fafa972c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:40:06.136333 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-37c9fce1-ecd2-4dcb-b5cd-760aac36ff29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:06.136961 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-37c9fce1-ecd2-4dcb-b5cd-760aac36ff29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:06.138479 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1302/2629] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:40:05 2022] DELETE /identity/v3/projects/d4f3cf0dcaf24043b7427f69fafa972c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:06.143436 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e42b6-a502-4b1f-8a13-1dff77207d6d None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:06.143937 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e42b6-a502-4b1f-8a13-1dff77207d6d None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:06.144037 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e42b6-a502-4b1f-8a13-1dff77207d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:06.163685 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-2f7e42b6-a502-4b1f-8a13-1dff77207d6d None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:06.164869 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-2f7e42b6-a502-4b1f-8a13-1dff77207d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:06.222014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1328/2630] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:40:06.230728 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d7fc0-4b15-4707-b967-c98a956d4a7a None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:06.231016 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d7fc0-4b15-4707-b967-c98a956d4a7a None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:06.231292 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d7fc0-4b15-4707-b967-c98a956d4a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:06.251891 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-676d7fc0-4b15-4707-b967-c98a956d4a7a None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:06.253127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-676d7fc0-4b15-4707-b967-c98a956d4a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:06.263803 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1303/2631] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:40:06.268232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e42b6-a502-4b1f-8a13-1dff77207d6d None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:06.278951 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e42b6-a502-4b1f-8a13-1dff77207d6d None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:06.283532 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb63952-77c8-40bd-b354-887e367cff22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:06.284133 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb63952-77c8-40bd-b354-887e367cff22 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:06.284411 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb63952-77c8-40bd-b354-887e367cff22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:06.284681 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb63952-77c8-40bd-b354-887e367cff22 admin admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:06.285323 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb63952-77c8-40bd-b354-887e367cff22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:40:06.285624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb63952-77c8-40bd-b354-887e367cff22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:06.287254 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb63952-77c8-40bd-b354-887e367cff22 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:06.299173 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1329/2632] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:40:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:40:06.303583 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d7fc0-4b15-4707-b967-c98a956d4a7a None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:06.313281 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d7fc0-4b15-4707-b967-c98a956d4a7a None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:06.317950 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560fa37-6d5b-4aa6-8405-e2cb250bfb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:06.318581 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560fa37-6d5b-4aa6-8405-e2cb250bfb8a admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:06.318864 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560fa37-6d5b-4aa6-8405-e2cb250bfb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:06.319157 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560fa37-6d5b-4aa6-8405-e2cb250bfb8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:06.319852 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7560fa37-6d5b-4aa6-8405-e2cb250bfb8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:06.322331 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7560fa37-6d5b-4aa6-8405-e2cb250bfb8a admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:06.349249 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1304/2633] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:40:06 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:40:06.353516 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb63952-77c8-40bd-b354-887e367cff22 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:06.363825 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb63952-77c8-40bd-b354-887e367cff22 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:06.420792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732801f3-60a8-41f8-9fc6-e1a456b298e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:06.421394 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732801f3-60a8-41f8-9fc6-e1a456b298e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:06.421672 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732801f3-60a8-41f8-9fc6-e1a456b298e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:06.421941 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732801f3-60a8-41f8-9fc6-e1a456b298e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:06.422605 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732801f3-60a8-41f8-9fc6-e1a456b298e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:06.424483 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732801f3-60a8-41f8-9fc6-e1a456b298e5 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:06.445396 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1330/2634] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:40:06 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:40:06.450014 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560fa37-6d5b-4aa6-8405-e2cb250bfb8a admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:06.460288 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560fa37-6d5b-4aa6-8405-e2cb250bfb8a admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:06.464992 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2cce4f-14c6-45af-89e1-fa006ac6c622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:06.465580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2cce4f-14c6-45af-89e1-fa006ac6c622 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:06.465850 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2cce4f-14c6-45af-89e1-fa006ac6c622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:06.466161 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2cce4f-14c6-45af-89e1-fa006ac6c622 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:06.466796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2cce4f-14c6-45af-89e1-fa006ac6c622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:06.468379 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2cce4f-14c6-45af-89e1-fa006ac6c622 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:06.478682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1305/2635] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:40:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:40:06.482769 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732801f3-60a8-41f8-9fc6-e1a456b298e5 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:06.492466 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732801f3-60a8-41f8-9fc6-e1a456b298e5 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:06.497605 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab68c1f0-d722-41b3-8253-fde5214b5e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:06.498214 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab68c1f0-d722-41b3-8253-fde5214b5e0d admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:06.498492 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab68c1f0-d722-41b3-8253-fde5214b5e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:06.498761 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab68c1f0-d722-41b3-8253-fde5214b5e0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:06.499386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab68c1f0-d722-41b3-8253-fde5214b5e0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:06.500947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab68c1f0-d722-41b3-8253-fde5214b5e0d admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:06.511313 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1331/2636] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:40:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:40:06.515199 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2cce4f-14c6-45af-89e1-fa006ac6c622 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:06.525357 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2cce4f-14c6-45af-89e1-fa006ac6c622 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:06.529948 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc43911-481b-4ca7-9630-21bc37e7a507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:06.530620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc43911-481b-4ca7-9630-21bc37e7a507 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:06.530929 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc43911-481b-4ca7-9630-21bc37e7a507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:06.531204 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc43911-481b-4ca7-9630-21bc37e7a507 admin admin] PATH_INFO: `/v3/projects/c2f889cc68ce48b69ab8ba2fea733b3a/users/d43dd5bbcbf24705a0c89c5629c6c6d6/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:06.546664 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc43911-481b-4ca7-9630-21bc37e7a507 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2f889cc68ce48b69ab8ba2fea733b3a, user_id=d43dd5bbcbf24705a0c89c5629c6c6d6, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:06.549333 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc43911-481b-4ca7-9630-21bc37e7a507 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:06.562858 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1306/2637] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:40:06 2022] PUT /identity/v3/projects/c2f889cc68ce48b69ab8ba2fea733b3a/users/d43dd5bbcbf24705a0c89c5629c6c6d6/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:06.573116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efb36b-3ec9-4e83-b1e8-7969ced7135c None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:06.573416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efb36b-3ec9-4e83-b1e8-7969ced7135c None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:06.573685 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efb36b-3ec9-4e83-b1e8-7969ced7135c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:06.580706 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:06.604117 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-13efb36b-3ec9-4e83-b1e8-7969ced7135c None None] MFA Rules not processed for user `d43dd5bbcbf24705a0c89c5629c6c6d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:06.605326 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-13efb36b-3ec9-4e83-b1e8-7969ced7135c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:06.673379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1332/2638] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:06.677845 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc43911-481b-4ca7-9630-21bc37e7a507 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:06.739444 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc43911-481b-4ca7-9630-21bc37e7a507 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:06.744515 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a834e0-33d9-44f3-8b86-09932553089c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:06.745115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a834e0-33d9-44f3-8b86-09932553089c admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:06.745397 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a834e0-33d9-44f3-8b86-09932553089c admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:06.745671 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a834e0-33d9-44f3-8b86-09932553089c admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:06.746342 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a834e0-33d9-44f3-8b86-09932553089c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:06.748231 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a834e0-33d9-44f3-8b86-09932553089c admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:06.775228 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1307/2639] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:40:06 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:40:06.779508 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efb36b-3ec9-4e83-b1e8-7969ced7135c None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:06.790130 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efb36b-3ec9-4e83-b1e8-7969ced7135c None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:06.846251 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcce526-eafe-4324-ae31-34005e81b8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:06.846861 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcce526-eafe-4324-ae31-34005e81b8c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:06.847135 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcce526-eafe-4324-ae31-34005e81b8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:06.847403 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcce526-eafe-4324-ae31-34005e81b8c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:06.848036 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcce526-eafe-4324-ae31-34005e81b8c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:06.849929 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcce526-eafe-4324-ae31-34005e81b8c4 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:06.870927 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1333/2640] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:40:06 2022] POST /identity/v3/users => generated 298 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:40:06.874989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a834e0-33d9-44f3-8b86-09932553089c admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:06.885693 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a834e0-33d9-44f3-8b86-09932553089c admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:06.890471 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2648b3d-ee15-4206-acce-6d750c270c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:06.891065 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2648b3d-ee15-4206-acce-6d750c270c41 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:06.891338 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2648b3d-ee15-4206-acce-6d750c270c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:06.891606 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2648b3d-ee15-4206-acce-6d750c270c41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:06.892549 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2648b3d-ee15-4206-acce-6d750c270c41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:06.893835 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2648b3d-ee15-4206-acce-6d750c270c41 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:06.904691 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1308/2641] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:40:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:40:06.908716 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcce526-eafe-4324-ae31-34005e81b8c4 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:06.918729 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcce526-eafe-4324-ae31-34005e81b8c4 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:06.923258 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f56baaf-e096-4b01-a297-d94efa386bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:06.923904 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56baaf-e096-4b01-a297-d94efa386bad admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:06.924172 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56baaf-e096-4b01-a297-d94efa386bad admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:06.924441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56baaf-e096-4b01-a297-d94efa386bad admin admin] PATH_INFO: `/v3/projects/277ef17ce7ed49a9b4a3056b7ace9c4b/users/c18b09e2d8094794b7d1360f7d0980a8/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:06.940132 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f56baaf-e096-4b01-a297-d94efa386bad admin admin] RBAC: Authorizing `identity:create_grant(project_id=277ef17ce7ed49a9b4a3056b7ace9c4b, user_id=c18b09e2d8094794b7d1360f7d0980a8, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:06.942834 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f56baaf-e096-4b01-a297-d94efa386bad admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:06.959083 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1334/2642] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:40:06 2022] PUT /identity/v3/projects/277ef17ce7ed49a9b4a3056b7ace9c4b/users/c18b09e2d8094794b7d1360f7d0980a8/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:40:06.963375 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2648b3d-ee15-4206-acce-6d750c270c41 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:06.973626 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2648b3d-ee15-4206-acce-6d750c270c41 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:06.986519 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:07.037287 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f33ee9-873c-42df-b37c-156dd4a61ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:07.037901 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f33ee9-873c-42df-b37c-156dd4a61ddd admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:07.038227 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f33ee9-873c-42df-b37c-156dd4a61ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:07.038498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f33ee9-873c-42df-b37c-156dd4a61ddd admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:07.039139 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f33ee9-873c-42df-b37c-156dd4a61ddd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:07.041328 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f33ee9-873c-42df-b37c-156dd4a61ddd admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:07.051636 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1309/2643] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:40:06 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:40:07.056444 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f56baaf-e096-4b01-a297-d94efa386bad admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:07.066649 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f56baaf-e096-4b01-a297-d94efa386bad admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:07.071342 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2987d8e8-97d7-4aad-b8e8-f1fdacf9aadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:07.071986 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987d8e8-97d7-4aad-b8e8-f1fdacf9aadd admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:07.072261 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987d8e8-97d7-4aad-b8e8-f1fdacf9aadd admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:07.072532 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987d8e8-97d7-4aad-b8e8-f1fdacf9aadd admin admin] PATH_INFO: `/v3/projects/277ef17ce7ed49a9b4a3056b7ace9c4b/users/c18b09e2d8094794b7d1360f7d0980a8/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:07.074460 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2987d8e8-97d7-4aad-b8e8-f1fdacf9aadd admin admin] RBAC: Authorizing `identity:create_grant(project_id=277ef17ce7ed49a9b4a3056b7ace9c4b, user_id=c18b09e2d8094794b7d1360f7d0980a8, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:07.077073 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2987d8e8-97d7-4aad-b8e8-f1fdacf9aadd admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:07.087371 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1335/2644] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:40:07 2022] PUT /identity/v3/projects/277ef17ce7ed49a9b4a3056b7ace9c4b/users/c18b09e2d8094794b7d1360f7d0980a8/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:07.098607 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd7a10b-d1f8-4cf4-b6f2-47ca4823e9e8 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:07.098904 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd7a10b-d1f8-4cf4-b6f2-47ca4823e9e8 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:07.099479 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd7a10b-d1f8-4cf4-b6f2-47ca4823e9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:07.129069 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-0dd7a10b-d1f8-4cf4-b6f2-47ca4823e9e8 None None] MFA Rules not processed for user `c18b09e2d8094794b7d1360f7d0980a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:07.130332 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-0dd7a10b-d1f8-4cf4-b6f2-47ca4823e9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:07.200799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1310/2645] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:07 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:07.212472 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2987d8e8-97d7-4aad-b8e8-f1fdacf9aadd admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:07.214287 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-2987d8e8-97d7-4aad-b8e8-f1fdacf9aadd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:07.263281 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:07.275401 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2987d8e8-97d7-4aad-b8e8-f1fdacf9aadd admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:07.279120 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a16ae-6709-4d41-8903-9f24ea36ad4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:07.279758 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a16ae-6709-4d41-8903-9f24ea36ad4c service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:07.280040 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a16ae-6709-4d41-8903-9f24ea36ad4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:07.280319 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a16ae-6709-4d41-8903-9f24ea36ad4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:07.290550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a16ae-6709-4d41-8903-9f24ea36ad4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:07.292001 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a16ae-6709-4d41-8903-9f24ea36ad4c service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:07.302230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1336/2646] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:40:07 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:40:07.374685 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd7a10b-d1f8-4cf4-b6f2-47ca4823e9e8 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:07.384240 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd7a10b-d1f8-4cf4-b6f2-47ca4823e9e8 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:07.387825 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5cd54c-f039-4dee-a175-637d7946d810 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:07.388451 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cd54c-f039-4dee-a175-637d7946d810 service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:07.388728 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cd54c-f039-4dee-a175-637d7946d810 service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:07.389000 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cd54c-f039-4dee-a175-637d7946d810 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:07.446900 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5cd54c-f039-4dee-a175-637d7946d810 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:07.448347 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5cd54c-f039-4dee-a175-637d7946d810 service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:07.457719 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1311/2647] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:40:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:40:07.728175 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a16ae-6709-4d41-8903-9f24ea36ad4c service cinder] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:07.790130 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a16ae-6709-4d41-8903-9f24ea36ad4c service cinder] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:07.794684 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b053db0-ad3c-4536-a175-b4f1de9ce84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:07.795321 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b053db0-ad3c-4536-a175-b4f1de9ce84b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:07.795599 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b053db0-ad3c-4536-a175-b4f1de9ce84b admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:07.795870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b053db0-ad3c-4536-a175-b4f1de9ce84b admin admin] PATH_INFO: `/v3/users/d43dd5bbcbf24705a0c89c5629c6c6d6` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:07.796992 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b053db0-ad3c-4536-a175-b4f1de9ce84b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d43dd5bbcbf24705a0c89c5629c6c6d6)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:07.798950 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b053db0-ad3c-4536-a175-b4f1de9ce84b admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:07.845900 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-2b053db0-ad3c-4536-a175-b4f1de9ce84b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd43dd5bbcbf24705a0c89c5629c6c6d6'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:07.846581 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-2b053db0-ad3c-4536-a175-b4f1de9ce84b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd43dd5bbcbf24705a0c89c5629c6c6d6'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:07.853379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-2b053db0-ad3c-4536-a175-b4f1de9ce84b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd43dd5bbcbf24705a0c89c5629c6c6d6'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:07.865181 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-2b053db0-ad3c-4536-a175-b4f1de9ce84b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd43dd5bbcbf24705a0c89c5629c6c6d6'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:07.865810 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-2b053db0-ad3c-4536-a175-b4f1de9ce84b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd43dd5bbcbf24705a0c89c5629c6c6d6'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:07.877555 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1337/2648] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:40:07 2022] DELETE /identity/v3/users/d43dd5bbcbf24705a0c89c5629c6c6d6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:07.881970 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1cb87-20c4-4f98-8e05-84013b238032 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:07.882456 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1cb87-20c4-4f98-8e05-84013b238032 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:07.882575 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1cb87-20c4-4f98-8e05-84013b238032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:07.902008 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-d0a1cb87-20c4-4f98-8e05-84013b238032 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:07.903228 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-d0a1cb87-20c4-4f98-8e05-84013b238032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:07.934970 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:07.961407 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1312/2649] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:07.971620 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b053db0-ad3c-4536-a175-b4f1de9ce84b admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:07.973018 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-2b053db0-ad3c-4536-a175-b4f1de9ce84b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:08.032055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b053db0-ad3c-4536-a175-b4f1de9ce84b admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:08.035677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ab3ae-83e7-4a2a-b193-80992ecb1ae5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:08.036282 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ab3ae-83e7-4a2a-b193-80992ecb1ae5 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:08.036563 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ab3ae-83e7-4a2a-b193-80992ecb1ae5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:08.036839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ab3ae-83e7-4a2a-b193-80992ecb1ae5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:08.046484 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ab3ae-83e7-4a2a-b193-80992ecb1ae5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:08.048215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ab3ae-83e7-4a2a-b193-80992ecb1ae5 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:08.058827 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1338/2650] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:40:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:40:08.350600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1cb87-20c4-4f98-8e05-84013b238032 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:08.352314 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-d0a1cb87-20c4-4f98-8e05-84013b238032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:08.371685 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1cb87-20c4-4f98-8e05-84013b238032 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:08.380775 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04824d1-3be6-4f90-bed8-e238dd94b720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:08.382208 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04824d1-3be6-4f90-bed8-e238dd94b720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:08.382410 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04824d1-3be6-4f90-bed8-e238dd94b720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:08.383249 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04824d1-3be6-4f90-bed8-e238dd94b720 admin admin] PATH_INFO: `/v3/projects/c2f889cc68ce48b69ab8ba2fea733b3a` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:08.388260 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04824d1-3be6-4f90-bed8-e238dd94b720 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2f889cc68ce48b69ab8ba2fea733b3a)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:08.390705 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04824d1-3be6-4f90-bed8-e238dd94b720 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:08.441038 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e04824d1-3be6-4f90-bed8-e238dd94b720 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2f889cc68ce48b69ab8ba2fea733b3a', 'request_id': 'req-e04824d1-3be6-4f90-bed8-e238dd94b720'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:08.448113 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e04824d1-3be6-4f90-bed8-e238dd94b720 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2f889cc68ce48b69ab8ba2fea733b3a', 'request_id': 'req-e04824d1-3be6-4f90-bed8-e238dd94b720'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:08.454495 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e04824d1-3be6-4f90-bed8-e238dd94b720 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2f889cc68ce48b69ab8ba2fea733b3a', 'request_id': 'req-e04824d1-3be6-4f90-bed8-e238dd94b720'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:08.468226 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e04824d1-3be6-4f90-bed8-e238dd94b720 admin admin] The token cache is being invalidate because project c2f889cc68ce48b69ab8ba2fea733b3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:40:08.468516 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e04824d1-3be6-4f90-bed8-e238dd94b720 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:08.469168 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-e04824d1-3be6-4f90-bed8-e238dd94b720 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:08.470684 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1313/2651] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:40:08 2022] DELETE /identity/v3/projects/c2f889cc68ce48b69ab8ba2fea733b3a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:08.474611 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ab3ae-83e7-4a2a-b193-80992ecb1ae5 service neutron] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:08.485309 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ab3ae-83e7-4a2a-b193-80992ecb1ae5 service neutron] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:08.489933 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:08.549267 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d0c3e-5fd6-4f7e-9bc3-10bb9bb6fa83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:08.549837 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d0c3e-5fd6-4f7e-9bc3-10bb9bb6fa83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:08.550151 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d0c3e-5fd6-4f7e-9bc3-10bb9bb6fa83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:08.550428 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d0c3e-5fd6-4f7e-9bc3-10bb9bb6fa83 admin admin] PATH_INFO: `/v3/users/c18b09e2d8094794b7d1360f7d0980a8` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:08.551550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5d0c3e-5fd6-4f7e-9bc3-10bb9bb6fa83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c18b09e2d8094794b7d1360f7d0980a8)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:08.553469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5d0c3e-5fd6-4f7e-9bc3-10bb9bb6fa83 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:08.601376 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-8e5d0c3e-5fd6-4f7e-9bc3-10bb9bb6fa83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c18b09e2d8094794b7d1360f7d0980a8'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:08.602003 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-8e5d0c3e-5fd6-4f7e-9bc3-10bb9bb6fa83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c18b09e2d8094794b7d1360f7d0980a8'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:08.611108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-8e5d0c3e-5fd6-4f7e-9bc3-10bb9bb6fa83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c18b09e2d8094794b7d1360f7d0980a8'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:08.628051 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-8e5d0c3e-5fd6-4f7e-9bc3-10bb9bb6fa83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c18b09e2d8094794b7d1360f7d0980a8'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:08.628727 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-8e5d0c3e-5fd6-4f7e-9bc3-10bb9bb6fa83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c18b09e2d8094794b7d1360f7d0980a8'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:08.641447 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1339/2652] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:40:08 2022] DELETE /identity/v3/users/c18b09e2d8094794b7d1360f7d0980a8 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:08.929792 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04824d1-3be6-4f90-bed8-e238dd94b720 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:08.931302 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-e04824d1-3be6-4f90-bed8-e238dd94b720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:08.944812 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04824d1-3be6-4f90-bed8-e238dd94b720 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:09.001271 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc05bc2-cc3c-4906-ac6b-bd7e592aa366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:09.001894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc05bc2-cc3c-4906-ac6b-bd7e592aa366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:09.002207 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc05bc2-cc3c-4906-ac6b-bd7e592aa366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:09.002479 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc05bc2-cc3c-4906-ac6b-bd7e592aa366 admin admin] PATH_INFO: `/v3/projects/277ef17ce7ed49a9b4a3056b7ace9c4b` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:09.003680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc05bc2-cc3c-4906-ac6b-bd7e592aa366 admin admin] RBAC: Authorizing `identity:delete_project(project_id=277ef17ce7ed49a9b4a3056b7ace9c4b)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:09.005661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc05bc2-cc3c-4906-ac6b-bd7e592aa366 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:09.052255 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-ecc05bc2-cc3c-4906-ac6b-bd7e592aa366 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '277ef17ce7ed49a9b4a3056b7ace9c4b', 'request_id': 'req-ecc05bc2-cc3c-4906-ac6b-bd7e592aa366'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:09.058822 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-ecc05bc2-cc3c-4906-ac6b-bd7e592aa366 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '277ef17ce7ed49a9b4a3056b7ace9c4b', 'request_id': 'req-ecc05bc2-cc3c-4906-ac6b-bd7e592aa366'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:09.064305 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-ecc05bc2-cc3c-4906-ac6b-bd7e592aa366 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '277ef17ce7ed49a9b4a3056b7ace9c4b', 'request_id': 'req-ecc05bc2-cc3c-4906-ac6b-bd7e592aa366'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:09.065917 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:09.079717 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-ecc05bc2-cc3c-4906-ac6b-bd7e592aa366 admin admin] The token cache is being invalidate because project 277ef17ce7ed49a9b4a3056b7ace9c4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:40:09.080009 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-ecc05bc2-cc3c-4906-ac6b-bd7e592aa366 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:09.080630 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-ecc05bc2-cc3c-4906-ac6b-bd7e592aa366 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:09.082123 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1314/2653] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:40:08 2022] DELETE /identity/v3/projects/277ef17ce7ed49a9b4a3056b7ace9c4b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:40:09.087470 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe03d3-1347-425f-89bf-929ff4084696 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:09.087756 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe03d3-1347-425f-89bf-929ff4084696 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:09.088034 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe03d3-1347-425f-89bf-929ff4084696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:09.108188 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-affe03d3-1347-425f-89bf-929ff4084696 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:09.109394 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-affe03d3-1347-425f-89bf-929ff4084696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:09.167106 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1340/2654] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:40:09.174669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc8533-5945-4e1d-ba1d-808d2e299f76 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:09.174960 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc8533-5945-4e1d-ba1d-808d2e299f76 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:09.175237 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc8533-5945-4e1d-ba1d-808d2e299f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:09.194784 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-24cc8533-5945-4e1d-ba1d-808d2e299f76 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:09.196001 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-24cc8533-5945-4e1d-ba1d-808d2e299f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:09.203848 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1315/2655] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:09.207929 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe03d3-1347-425f-89bf-929ff4084696 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:09.218508 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe03d3-1347-425f-89bf-929ff4084696 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:09.223551 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee52889-0d86-4374-822d-24b078414911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:09.224272 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee52889-0d86-4374-822d-24b078414911 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:09.224616 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee52889-0d86-4374-822d-24b078414911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:09.224900 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee52889-0d86-4374-822d-24b078414911 admin admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:09.225545 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee52889-0d86-4374-822d-24b078414911 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:40:09.225850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee52889-0d86-4374-822d-24b078414911 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:09.228142 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee52889-0d86-4374-822d-24b078414911 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:09.244103 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1341/2656] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:40:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:40:09.245180 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc8533-5945-4e1d-ba1d-808d2e299f76 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:09.254549 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc8533-5945-4e1d-ba1d-808d2e299f76 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:09.259759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26fff8b-ab1f-45f4-aa00-87542513d1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:09.260352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26fff8b-ab1f-45f4-aa00-87542513d1a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:09.260637 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26fff8b-ab1f-45f4-aa00-87542513d1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:09.260905 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26fff8b-ab1f-45f4-aa00-87542513d1a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:09.261540 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26fff8b-ab1f-45f4-aa00-87542513d1a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:09.263460 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26fff8b-ab1f-45f4-aa00-87542513d1a9 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:09.291980 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1316/2657] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:40:09 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:40:09.296163 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee52889-0d86-4374-822d-24b078414911 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:09.306460 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee52889-0d86-4374-822d-24b078414911 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:09.319621 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:09.370248 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cbb037-4ee5-4212-adc0-4ad5468c071b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:09.370931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbb037-4ee5-4212-adc0-4ad5468c071b admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:09.371275 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbb037-4ee5-4212-adc0-4ad5468c071b admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:09.371617 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbb037-4ee5-4212-adc0-4ad5468c071b admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:09.372242 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cbb037-4ee5-4212-adc0-4ad5468c071b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:09.374162 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cbb037-4ee5-4212-adc0-4ad5468c071b admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:09.396356 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1342/2658] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:40:09 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:40:09.401010 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26fff8b-ab1f-45f4-aa00-87542513d1a9 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:09.411327 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26fff8b-ab1f-45f4-aa00-87542513d1a9 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:09.416014 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453c13f-bba3-49e8-8a49-addff83111e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:09.416611 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453c13f-bba3-49e8-8a49-addff83111e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:09.416893 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453c13f-bba3-49e8-8a49-addff83111e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:09.417167 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453c13f-bba3-49e8-8a49-addff83111e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:09.417809 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453c13f-bba3-49e8-8a49-addff83111e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:09.420536 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453c13f-bba3-49e8-8a49-addff83111e8 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:09.430756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1317/2659] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:40:09.434726 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cbb037-4ee5-4212-adc0-4ad5468c071b admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:09.445465 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cbb037-4ee5-4212-adc0-4ad5468c071b admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:09.451569 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29086e1-8996-4035-9429-8f82cc220e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:09.452241 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29086e1-8996-4035-9429-8f82cc220e56 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:09.452645 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29086e1-8996-4035-9429-8f82cc220e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:09.452933 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29086e1-8996-4035-9429-8f82cc220e56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:09.453608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29086e1-8996-4035-9429-8f82cc220e56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:09.455545 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29086e1-8996-4035-9429-8f82cc220e56 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:09.468061 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1343/2660] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:40:09.472291 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453c13f-bba3-49e8-8a49-addff83111e8 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:09.482897 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453c13f-bba3-49e8-8a49-addff83111e8 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:09.487724 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be484fdd-0536-4d16-b6ad-52f3483b2a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:09.488607 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be484fdd-0536-4d16-b6ad-52f3483b2a6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:09.489371 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be484fdd-0536-4d16-b6ad-52f3483b2a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:09.489371 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be484fdd-0536-4d16-b6ad-52f3483b2a6e admin admin] PATH_INFO: `/v3/projects/93a1e64bb68042beab682707aa82ed09/users/6f020a08c8d14d9897ac56ae1d24e6df/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:09.510059 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be484fdd-0536-4d16-b6ad-52f3483b2a6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=93a1e64bb68042beab682707aa82ed09, user_id=6f020a08c8d14d9897ac56ae1d24e6df, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:09.512734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be484fdd-0536-4d16-b6ad-52f3483b2a6e admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:09.526607 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1318/2661] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:40:09 2022] PUT /identity/v3/projects/93a1e64bb68042beab682707aa82ed09/users/6f020a08c8d14d9897ac56ae1d24e6df/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:09.538464 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3055cfe-bdb4-42bb-81f6-289c1fb31915 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:09.538943 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3055cfe-bdb4-42bb-81f6-289c1fb31915 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:09.539027 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3055cfe-bdb4-42bb-81f6-289c1fb31915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:09.577622 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-d3055cfe-bdb4-42bb-81f6-289c1fb31915 None None] MFA Rules not processed for user `6f020a08c8d14d9897ac56ae1d24e6df`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:09.579126 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-d3055cfe-bdb4-42bb-81f6-289c1fb31915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:09.672626 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1344/2662] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:09.684780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be484fdd-0536-4d16-b6ad-52f3483b2a6e admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:09.686607 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-be484fdd-0536-4d16-b6ad-52f3483b2a6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:09.748817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be484fdd-0536-4d16-b6ad-52f3483b2a6e admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:09.752344 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475b804-bc03-4875-b1ef-760f4deb5ced service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:09.752945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475b804-bc03-4875-b1ef-760f4deb5ced service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:09.753221 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475b804-bc03-4875-b1ef-760f4deb5ced service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:09.753486 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475b804-bc03-4875-b1ef-760f4deb5ced service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:09.762261 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d475b804-bc03-4875-b1ef-760f4deb5ced service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:09.763669 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d475b804-bc03-4875-b1ef-760f4deb5ced service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:09.772402 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1319/2663] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:40:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:40:09.791449 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3055cfe-bdb4-42bb-81f6-289c1fb31915 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:09.853872 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3055cfe-bdb4-42bb-81f6-289c1fb31915 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:09.858981 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39908f0-85ab-44ee-8a83-c047c180275c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:09.859604 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39908f0-85ab-44ee-8a83-c047c180275c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:09.859915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39908f0-85ab-44ee-8a83-c047c180275c admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:09.860202 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39908f0-85ab-44ee-8a83-c047c180275c admin admin] PATH_INFO: `/v3/users/6f020a08c8d14d9897ac56ae1d24e6df` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:09.861481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39908f0-85ab-44ee-8a83-c047c180275c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f020a08c8d14d9897ac56ae1d24e6df)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:09.863426 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39908f0-85ab-44ee-8a83-c047c180275c admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:09.892169 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:09.918149 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f39908f0-85ab-44ee-8a83-c047c180275c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f020a08c8d14d9897ac56ae1d24e6df'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:09.918816 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f39908f0-85ab-44ee-8a83-c047c180275c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f020a08c8d14d9897ac56ae1d24e6df'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:09.925963 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f39908f0-85ab-44ee-8a83-c047c180275c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f020a08c8d14d9897ac56ae1d24e6df'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:09.938567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f39908f0-85ab-44ee-8a83-c047c180275c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f020a08c8d14d9897ac56ae1d24e6df'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:09.939230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f39908f0-85ab-44ee-8a83-c047c180275c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f020a08c8d14d9897ac56ae1d24e6df'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:09.952242 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1345/2664] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:40:09 2022] DELETE /identity/v3/users/6f020a08c8d14d9897ac56ae1d24e6df => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:09.956725 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43e179-c223-46ef-a341-ba53171fac25 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:09.957183 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43e179-c223-46ef-a341-ba53171fac25 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:09.957332 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43e179-c223-46ef-a341-ba53171fac25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:09.976130 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-4b43e179-c223-46ef-a341-ba53171fac25 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:09.977331 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-4b43e179-c223-46ef-a341-ba53171fac25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:10.036264 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1320/2665] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:40:10.046647 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39908f0-85ab-44ee-8a83-c047c180275c admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:10.048038 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-f39908f0-85ab-44ee-8a83-c047c180275c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:10.110638 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39908f0-85ab-44ee-8a83-c047c180275c admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:10.114176 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835337c2-1da0-4fc6-baa6-e0b9699d7e3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:10.114778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835337c2-1da0-4fc6-baa6-e0b9699d7e3e service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:10.115054 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835337c2-1da0-4fc6-baa6-e0b9699d7e3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:10.115454 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835337c2-1da0-4fc6-baa6-e0b9699d7e3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:10.127097 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835337c2-1da0-4fc6-baa6-e0b9699d7e3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:10.128502 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835337c2-1da0-4fc6-baa6-e0b9699d7e3e service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:10.138350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1346/2666] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:40:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:40:10.411867 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43e179-c223-46ef-a341-ba53171fac25 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:10.413288 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-4b43e179-c223-46ef-a341-ba53171fac25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:10.427466 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43e179-c223-46ef-a341-ba53171fac25 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:10.432896 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4eba7-a65b-4fa6-983e-a26e7cc17a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:10.433631 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4eba7-a65b-4fa6-983e-a26e7cc17a14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:10.433980 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4eba7-a65b-4fa6-983e-a26e7cc17a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:10.434353 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4eba7-a65b-4fa6-983e-a26e7cc17a14 admin admin] PATH_INFO: `/v3/projects/93a1e64bb68042beab682707aa82ed09` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:10.435646 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd4eba7-a65b-4fa6-983e-a26e7cc17a14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93a1e64bb68042beab682707aa82ed09)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:10.437592 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd4eba7-a65b-4fa6-983e-a26e7cc17a14 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:10.474264 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:INSERT=3,keystone:DELETE=2,keystone:SELECT=95 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:10.496065 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-edd4eba7-a65b-4fa6-983e-a26e7cc17a14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93a1e64bb68042beab682707aa82ed09', 'request_id': 'req-edd4eba7-a65b-4fa6-983e-a26e7cc17a14'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:10.502452 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-edd4eba7-a65b-4fa6-983e-a26e7cc17a14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93a1e64bb68042beab682707aa82ed09', 'request_id': 'req-edd4eba7-a65b-4fa6-983e-a26e7cc17a14'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:10.508519 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-edd4eba7-a65b-4fa6-983e-a26e7cc17a14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93a1e64bb68042beab682707aa82ed09', 'request_id': 'req-edd4eba7-a65b-4fa6-983e-a26e7cc17a14'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:10.533354 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-edd4eba7-a65b-4fa6-983e-a26e7cc17a14 admin admin] The token cache is being invalidate because project 93a1e64bb68042beab682707aa82ed09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:40:10.533663 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-edd4eba7-a65b-4fa6-983e-a26e7cc17a14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:10.534321 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-edd4eba7-a65b-4fa6-983e-a26e7cc17a14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:10.538074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1321/2667] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:40:10 2022] DELETE /identity/v3/projects/93a1e64bb68042beab682707aa82ed09 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:10.546985 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0852b-eac9-4852-b871-c29dbaded1d7 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:10.547737 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0852b-eac9-4852-b871-c29dbaded1d7 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:10.547737 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0852b-eac9-4852-b871-c29dbaded1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:10.576388 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-f2f0852b-eac9-4852-b871-c29dbaded1d7 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:10.577608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-f2f0852b-eac9-4852-b871-c29dbaded1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:10.643745 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1347/2668] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:10.651971 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e0662-4714-4a7a-8bdb-d1ca75157bb1 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:10.652277 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e0662-4714-4a7a-8bdb-d1ca75157bb1 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:10.652514 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e0662-4714-4a7a-8bdb-d1ca75157bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:10.683697 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-eb0e0662-4714-4a7a-8bdb-d1ca75157bb1 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:10.684894 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-eb0e0662-4714-4a7a-8bdb-d1ca75157bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:10.694042 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1322/2669] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:10.698364 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f0852b-eac9-4852-b871-c29dbaded1d7 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:10.709137 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f0852b-eac9-4852-b871-c29dbaded1d7 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:10.714870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f0ffa-a55b-417a-857a-752570a00a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:10.715803 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f0ffa-a55b-417a-857a-752570a00a9c admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:10.716120 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f0ffa-a55b-417a-857a-752570a00a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:10.716395 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f0ffa-a55b-417a-857a-752570a00a9c admin admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:10.717046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595f0ffa-a55b-417a-857a-752570a00a9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:40:10.717333 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595f0ffa-a55b-417a-857a-752570a00a9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:10.718947 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595f0ffa-a55b-417a-857a-752570a00a9c admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:10.736336 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1348/2670] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:40:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:40:10.737668 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e0662-4714-4a7a-8bdb-d1ca75157bb1 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:10.751772 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e0662-4714-4a7a-8bdb-d1ca75157bb1 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:10.760585 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce11514-d832-448a-9f6e-41e61beb0c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:10.761645 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce11514-d832-448a-9f6e-41e61beb0c53 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:10.761961 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce11514-d832-448a-9f6e-41e61beb0c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:10.762845 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce11514-d832-448a-9f6e-41e61beb0c53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:10.763501 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce11514-d832-448a-9f6e-41e61beb0c53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:10.766745 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce11514-d832-448a-9f6e-41e61beb0c53 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:10.806147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1323/2671] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:40:10 2022] POST /identity/v3/projects => generated 375 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:40:10.810648 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f0ffa-a55b-417a-857a-752570a00a9c admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:10.834308 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f0ffa-a55b-417a-857a-752570a00a9c admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:10.876714 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:10.914222 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b8951-5790-4d26-9dc4-bdeacf46e8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:10.914762 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b8951-5790-4d26-9dc4-bdeacf46e8b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:10.915571 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b8951-5790-4d26-9dc4-bdeacf46e8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:10.915840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b8951-5790-4d26-9dc4-bdeacf46e8b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:10.916460 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b8951-5790-4d26-9dc4-bdeacf46e8b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:10.918999 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b8951-5790-4d26-9dc4-bdeacf46e8b7 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:10.941305 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1349/2672] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:40:10 2022] POST /identity/v3/users => generated 288 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:40:10.945488 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce11514-d832-448a-9f6e-41e61beb0c53 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:10.955749 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce11514-d832-448a-9f6e-41e61beb0c53 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:10.960440 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9e917-772c-4aca-953d-4e93673545d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:10.961039 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9e917-772c-4aca-953d-4e93673545d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:10.961411 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9e917-772c-4aca-953d-4e93673545d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:10.961712 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9e917-772c-4aca-953d-4e93673545d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:10.962431 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9e917-772c-4aca-953d-4e93673545d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:10.964346 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f9e917-772c-4aca-953d-4e93673545d0 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:10.974132 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1324/2673] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:40:10.978489 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b8951-5790-4d26-9dc4-bdeacf46e8b7 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:10.991096 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b8951-5790-4d26-9dc4-bdeacf46e8b7 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:10.996582 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb0f21-9228-4492-a871-b503182ff06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:10.997186 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb0f21-9228-4492-a871-b503182ff06a admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:10.997461 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb0f21-9228-4492-a871-b503182ff06a admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:10.997730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb0f21-9228-4492-a871-b503182ff06a admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:10.998412 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb0f21-9228-4492-a871-b503182ff06a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:11.000001 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb0f21-9228-4492-a871-b503182ff06a admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:11.012047 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1350/2674] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:40:11.017369 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9e917-772c-4aca-953d-4e93673545d0 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:11.028430 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9e917-772c-4aca-953d-4e93673545d0 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:11.033505 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b07741-ff84-4588-acc3-7d14912d765d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:11.034279 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b07741-ff84-4588-acc3-7d14912d765d admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:11.034559 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b07741-ff84-4588-acc3-7d14912d765d admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:11.034835 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b07741-ff84-4588-acc3-7d14912d765d admin admin] PATH_INFO: `/v3/projects/4c206e1031144f509a8957ad347ac345/users/1f631f56726c4cab8bd7d61076208b18/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:11.053906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b07741-ff84-4588-acc3-7d14912d765d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c206e1031144f509a8957ad347ac345, user_id=1f631f56726c4cab8bd7d61076208b18, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:11.057377 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b07741-ff84-4588-acc3-7d14912d765d admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:11.071403 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1325/2675] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:40:11 2022] PUT /identity/v3/projects/4c206e1031144f509a8957ad347ac345/users/1f631f56726c4cab8bd7d61076208b18/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:40:11.081898 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7a072-faae-4635-bc34-1310e0478ec2 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:11.082324 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7a072-faae-4635-bc34-1310e0478ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:11.082508 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7a072-faae-4635-bc34-1310e0478ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:11.113154 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-60a7a072-faae-4635-bc34-1310e0478ec2 None None] MFA Rules not processed for user `1f631f56726c4cab8bd7d61076208b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:11.114750 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-60a7a072-faae-4635-bc34-1310e0478ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:11.181810 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1351/2676] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:11.193344 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b07741-ff84-4588-acc3-7d14912d765d admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:11.195051 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-f1b07741-ff84-4588-acc3-7d14912d765d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:11.254932 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b07741-ff84-4588-acc3-7d14912d765d admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:11.258793 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2567d-d452-4328-8753-37a54f705273 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:11.259425 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2567d-d452-4328-8753-37a54f705273 service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:11.259702 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2567d-d452-4328-8753-37a54f705273 service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:11.260035 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2567d-d452-4328-8753-37a54f705273 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:11.268417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de2567d-d452-4328-8753-37a54f705273 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:11.269830 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de2567d-d452-4328-8753-37a54f705273 service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:11.279229 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1326/2677] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:40:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:40:17.977550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965336b9-0788-4e00-9919-3df4b073420a None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:17.978186 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965336b9-0788-4e00-9919-3df4b073420a None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:17.978300 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965336b9-0788-4e00-9919-3df4b073420a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:18.007901 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-965336b9-0788-4e00-9919-3df4b073420a None None] MFA Rules not processed for user `1f631f56726c4cab8bd7d61076208b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:18.009508 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-965336b9-0788-4e00-9919-3df4b073420a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:18.017813 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1352/2678] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:21.274708 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:DELETE=6,keystone:SELECT=67,keystone:INSERT=3 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:21.799514 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1d77e-cf6e-4b36-b62d-94cbe0d3f556 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:21.799804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1d77e-cf6e-4b36-b62d-94cbe0d3f556 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:21.800083 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1d77e-cf6e-4b36-b62d-94cbe0d3f556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:21.820080 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-d5b1d77e-cf6e-4b36-b62d-94cbe0d3f556 None None] MFA Rules not processed for user `57fa5ee68e6f48e982c312e6bb3a6ce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:21.821289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-d5b1d77e-cf6e-4b36-b62d-94cbe0d3f556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:21.878429 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1327/2679] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:22.186962 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965336b9-0788-4e00-9919-3df4b073420a None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:22.190192 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-965336b9-0788-4e00-9919-3df4b073420a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:22.273356 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965336b9-0788-4e00-9919-3df4b073420a None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:22.278913 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf612b-c236-4105-8b3d-241e233c6a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:22.280922 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf612b-c236-4105-8b3d-241e233c6a02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:22.281205 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf612b-c236-4105-8b3d-241e233c6a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:22.281479 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf612b-c236-4105-8b3d-241e233c6a02 admin admin] PATH_INFO: `/v3/users/6bd46cfb4c2c408780db90951d37c916` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:22.282691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf612b-c236-4105-8b3d-241e233c6a02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bd46cfb4c2c408780db90951d37c916)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:22.284609 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf612b-c236-4105-8b3d-241e233c6a02 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:22.341215 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:22.352253 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-8bdf612b-c236-4105-8b3d-241e233c6a02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bd46cfb4c2c408780db90951d37c916'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:22.352967 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-8bdf612b-c236-4105-8b3d-241e233c6a02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bd46cfb4c2c408780db90951d37c916'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:22.360661 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-8bdf612b-c236-4105-8b3d-241e233c6a02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bd46cfb4c2c408780db90951d37c916'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:22.373379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-8bdf612b-c236-4105-8b3d-241e233c6a02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bd46cfb4c2c408780db90951d37c916'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:22.374335 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-8bdf612b-c236-4105-8b3d-241e233c6a02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bd46cfb4c2c408780db90951d37c916'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:22.388332 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1353/2680] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:40:22 2022] DELETE /identity/v3/users/6bd46cfb4c2c408780db90951d37c916 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:22.392834 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75aacb-e21e-47bb-832b-4809b6f30726 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:22.393328 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75aacb-e21e-47bb-832b-4809b6f30726 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:22.393439 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75aacb-e21e-47bb-832b-4809b6f30726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:22.412521 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-8e75aacb-e21e-47bb-832b-4809b6f30726 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:22.413734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-8e75aacb-e21e-47bb-832b-4809b6f30726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:22.474217 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1328/2681] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:22.483695 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf612b-c236-4105-8b3d-241e233c6a02 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:22.485113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-8bdf612b-c236-4105-8b3d-241e233c6a02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:22.560054 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf612b-c236-4105-8b3d-241e233c6a02 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:22.564824 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e16afe3-37e5-4d4b-8830-cbee5facc471 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:22.565122 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16afe3-37e5-4d4b-8830-cbee5facc471 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:22.565399 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16afe3-37e5-4d4b-8830-cbee5facc471 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:22.565668 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16afe3-37e5-4d4b-8830-cbee5facc471 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:22.581879 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e16afe3-37e5-4d4b-8830-cbee5facc471 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:22.585880 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e16afe3-37e5-4d4b-8830-cbee5facc471 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:22.611883 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1354/2682] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:40:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:40:23.055056 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75aacb-e21e-47bb-832b-4809b6f30726 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:23.056943 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-8e75aacb-e21e-47bb-832b-4809b6f30726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:23.087884 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75aacb-e21e-47bb-832b-4809b6f30726 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:23.095467 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2378bd-5a4e-454e-91dd-6fed93665905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:23.096388 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2378bd-5a4e-454e-91dd-6fed93665905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:23.096874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2378bd-5a4e-454e-91dd-6fed93665905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:23.097145 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2378bd-5a4e-454e-91dd-6fed93665905 admin admin] PATH_INFO: `/v3/projects/cfe6d038da334f89a92ef46bbbf6d5d3` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:23.098330 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2378bd-5a4e-454e-91dd-6fed93665905 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfe6d038da334f89a92ef46bbbf6d5d3)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:23.100609 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2378bd-5a4e-454e-91dd-6fed93665905 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:23.167111 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-0f2378bd-5a4e-454e-91dd-6fed93665905 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfe6d038da334f89a92ef46bbbf6d5d3', 'request_id': 'req-0f2378bd-5a4e-454e-91dd-6fed93665905'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:23.175879 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-0f2378bd-5a4e-454e-91dd-6fed93665905 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfe6d038da334f89a92ef46bbbf6d5d3', 'request_id': 'req-0f2378bd-5a4e-454e-91dd-6fed93665905'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:23.184529 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-0f2378bd-5a4e-454e-91dd-6fed93665905 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfe6d038da334f89a92ef46bbbf6d5d3', 'request_id': 'req-0f2378bd-5a4e-454e-91dd-6fed93665905'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:23.203705 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-0f2378bd-5a4e-454e-91dd-6fed93665905 admin admin] The token cache is being invalidate because project cfe6d038da334f89a92ef46bbbf6d5d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:40:23.204035 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-0f2378bd-5a4e-454e-91dd-6fed93665905 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:23.204666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-0f2378bd-5a4e-454e-91dd-6fed93665905 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:23.209762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1329/2683] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:40:23 2022] DELETE /identity/v3/projects/cfe6d038da334f89a92ef46bbbf6d5d3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:23.211970 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257a43f-f84f-430c-872c-a9e7abbad8ac None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:23.212419 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257a43f-f84f-430c-872c-a9e7abbad8ac None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:23.212523 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257a43f-f84f-430c-872c-a9e7abbad8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:23.233776 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-7257a43f-f84f-430c-872c-a9e7abbad8ac None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:23.235189 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-7257a43f-f84f-430c-872c-a9e7abbad8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:23.304965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1355/2684] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:23.313349 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6c5f0-bc77-4c78-9fcf-f9358b4f44ee None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:23.313956 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6c5f0-bc77-4c78-9fcf-f9358b4f44ee None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:23.314283 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6c5f0-bc77-4c78-9fcf-f9358b4f44ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:23.336414 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-e0a6c5f0-bc77-4c78-9fcf-f9358b4f44ee None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:23.337897 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-e0a6c5f0-bc77-4c78-9fcf-f9358b4f44ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:23.350710 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1330/2685] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:40:23.351820 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257a43f-f84f-430c-872c-a9e7abbad8ac None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:23.362903 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257a43f-f84f-430c-872c-a9e7abbad8ac None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:23.372014 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0719b531-838a-4a33-9d21-6ec5bc129a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:23.372889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719b531-838a-4a33-9d21-6ec5bc129a44 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:23.373335 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719b531-838a-4a33-9d21-6ec5bc129a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:23.373608 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719b531-838a-4a33-9d21-6ec5bc129a44 admin admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:23.374312 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0719b531-838a-4a33-9d21-6ec5bc129a44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:40:23.374822 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0719b531-838a-4a33-9d21-6ec5bc129a44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:23.376611 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0719b531-838a-4a33-9d21-6ec5bc129a44 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:23.394342 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1356/2686] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:40:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:40:23.395956 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a6c5f0-bc77-4c78-9fcf-f9358b4f44ee None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:23.406154 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a6c5f0-bc77-4c78-9fcf-f9358b4f44ee None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:23.411230 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2488023b-9d7f-4134-970a-03cabbfc431d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:23.412269 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488023b-9d7f-4134-970a-03cabbfc431d admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:23.412589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488023b-9d7f-4134-970a-03cabbfc431d admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:23.412866 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488023b-9d7f-4134-970a-03cabbfc431d admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:23.413729 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2488023b-9d7f-4134-970a-03cabbfc431d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:23.415473 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2488023b-9d7f-4134-970a-03cabbfc431d admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:23.442006 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:23.451327 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1331/2687] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:40:23 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:40:23.455253 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0719b531-838a-4a33-9d21-6ec5bc129a44 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:23.467008 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0719b531-838a-4a33-9d21-6ec5bc129a44 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:23.535137 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:23.560685 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d824f-a19e-47c9-9b60-53926f210b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:23.561442 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d824f-a19e-47c9-9b60-53926f210b1a admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:23.561914 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d824f-a19e-47c9-9b60-53926f210b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:23.562520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d824f-a19e-47c9-9b60-53926f210b1a admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:23.563162 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d824f-a19e-47c9-9b60-53926f210b1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:23.565787 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d824f-a19e-47c9-9b60-53926f210b1a admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:23.592786 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1357/2688] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:40:23 2022] POST /identity/v3/users => generated 287 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:40:23.596934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2488023b-9d7f-4134-970a-03cabbfc431d admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:23.612088 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2488023b-9d7f-4134-970a-03cabbfc431d admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:23.618641 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a349140e-e6b7-4720-a6fe-7d4b7fc2a337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:23.619969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349140e-e6b7-4720-a6fe-7d4b7fc2a337 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:23.620313 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349140e-e6b7-4720-a6fe-7d4b7fc2a337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:23.620666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349140e-e6b7-4720-a6fe-7d4b7fc2a337 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:23.621308 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a349140e-e6b7-4720-a6fe-7d4b7fc2a337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:23.622992 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a349140e-e6b7-4720-a6fe-7d4b7fc2a337 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:23.635761 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1332/2689] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:40:23.640848 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d824f-a19e-47c9-9b60-53926f210b1a admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:23.654445 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d824f-a19e-47c9-9b60-53926f210b1a admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:23.660277 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ef6a8-3bb8-4b4f-b107-1bc4e3511761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:23.661460 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ef6a8-3bb8-4b4f-b107-1bc4e3511761 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:23.661460 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ef6a8-3bb8-4b4f-b107-1bc4e3511761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:23.661693 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ef6a8-3bb8-4b4f-b107-1bc4e3511761 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:23.662618 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ef6a8-3bb8-4b4f-b107-1bc4e3511761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:23.664901 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ef6a8-3bb8-4b4f-b107-1bc4e3511761 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:23.682444 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1358/2690] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:40:23.686630 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a349140e-e6b7-4720-a6fe-7d4b7fc2a337 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:23.700794 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a349140e-e6b7-4720-a6fe-7d4b7fc2a337 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:23.707880 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd56618-ec10-4d8b-ad15-b85b643e1119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:23.708544 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd56618-ec10-4d8b-ad15-b85b643e1119 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:23.708817 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd56618-ec10-4d8b-ad15-b85b643e1119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:23.709084 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd56618-ec10-4d8b-ad15-b85b643e1119 admin admin] PATH_INFO: `/v3/projects/0ee33b2a73604a70bc5e0bd14d1ff109/users/5c5ba905177e4f71908396ba3689f47d/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:23.852724 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd56618-ec10-4d8b-ad15-b85b643e1119 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ee33b2a73604a70bc5e0bd14d1ff109, user_id=5c5ba905177e4f71908396ba3689f47d, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:23.855446 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd56618-ec10-4d8b-ad15-b85b643e1119 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:23.883693 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1333/2691] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:40:23 2022] PUT /identity/v3/projects/0ee33b2a73604a70bc5e0bd14d1ff109/users/5c5ba905177e4f71908396ba3689f47d/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:23.889195 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99d75c-d576-4b14-b800-91d9c69193f3 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:23.889481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99d75c-d576-4b14-b800-91d9c69193f3 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:23.889755 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99d75c-d576-4b14-b800-91d9c69193f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:23.949516 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-0d99d75c-d576-4b14-b800-91d9c69193f3 None None] MFA Rules not processed for user `5c5ba905177e4f71908396ba3689f47d`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:23.951318 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-0d99d75c-d576-4b14-b800-91d9c69193f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:24.079327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1359/2692] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:24.085980 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd56618-ec10-4d8b-ad15-b85b643e1119 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:24.087776 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-bcd56618-ec10-4d8b-ad15-b85b643e1119 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:24.187762 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd56618-ec10-4d8b-ad15-b85b643e1119 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:24.193862 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a5e65-635b-4511-802a-a3f8e545cc7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:24.194989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a5e65-635b-4511-802a-a3f8e545cc7e service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:24.195271 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a5e65-635b-4511-802a-a3f8e545cc7e service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:24.195544 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a5e65-635b-4511-802a-a3f8e545cc7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:24.208774 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a5e65-635b-4511-802a-a3f8e545cc7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:24.210466 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a5e65-635b-4511-802a-a3f8e545cc7e service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:24.221586 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1334/2693] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:40:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:40:26.483919 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d99d75c-d576-4b14-b800-91d9c69193f3 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:26.485392 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-0d99d75c-d576-4b14-b800-91d9c69193f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:26.555228 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d99d75c-d576-4b14-b800-91d9c69193f3 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:26.559230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5513fe12-11c9-4960-ae59-e5ebdb929832 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:26.559850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5513fe12-11c9-4960-ae59-e5ebdb929832 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:26.560132 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5513fe12-11c9-4960-ae59-e5ebdb929832 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:26.560410 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5513fe12-11c9-4960-ae59-e5ebdb929832 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:26.562223 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-5513fe12-11c9-4960-ae59-e5ebdb929832 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:26.632551 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5513fe12-11c9-4960-ae59-e5ebdb929832 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:26.634001 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5513fe12-11c9-4960-ae59-e5ebdb929832 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:26.645148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1360/2694] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:40:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:40:30.275528 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a5e65-635b-4511-802a-a3f8e545cc7e service cinder] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:30.278726 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-2a1a5e65-635b-4511-802a-a3f8e545cc7e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:30.307117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a5e65-635b-4511-802a-a3f8e545cc7e service cinder] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:30.312887 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad9b6e-56fe-4690-b835-a2412bda1771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:30.313525 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad9b6e-56fe-4690-b835-a2412bda1771 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:30.314171 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad9b6e-56fe-4690-b835-a2412bda1771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:30.314661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad9b6e-56fe-4690-b835-a2412bda1771 admin admin] PATH_INFO: `/v3/users/8c8ebfdad6ab437eb7dda111fee84205` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:30.317066 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad9b6e-56fe-4690-b835-a2412bda1771 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c8ebfdad6ab437eb7dda111fee84205)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:30.320336 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad9b6e-56fe-4690-b835-a2412bda1771 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:30.410555 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-b7ad9b6e-56fe-4690-b835-a2412bda1771 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c8ebfdad6ab437eb7dda111fee84205'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:30.411297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-b7ad9b6e-56fe-4690-b835-a2412bda1771 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c8ebfdad6ab437eb7dda111fee84205'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:30.411991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-b7ad9b6e-56fe-4690-b835-a2412bda1771 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c8ebfdad6ab437eb7dda111fee84205'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:30.420639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-b7ad9b6e-56fe-4690-b835-a2412bda1771 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c8ebfdad6ab437eb7dda111fee84205'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:30.434172 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-b7ad9b6e-56fe-4690-b835-a2412bda1771 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c8ebfdad6ab437eb7dda111fee84205'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:30.458456 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1335/2695] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:40:30 2022] DELETE /identity/v3/users/8c8ebfdad6ab437eb7dda111fee84205 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:30.644217 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5513fe12-11c9-4960-ae59-e5ebdb929832 service neutron] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:30.650121 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:30.660449 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5513fe12-11c9-4960-ae59-e5ebdb929832 service neutron] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:30.720916 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18230fb-d675-462f-bc27-7acafd91b68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:30.721550 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18230fb-d675-462f-bc27-7acafd91b68d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:30.721829 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18230fb-d675-462f-bc27-7acafd91b68d admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:30.722126 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18230fb-d675-462f-bc27-7acafd91b68d admin admin] PATH_INFO: `/v3/projects/38cccae871b94a9db82b5017c2c497c1` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:30.723251 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18230fb-d675-462f-bc27-7acafd91b68d admin admin] RBAC: Authorizing `identity:delete_project(project_id=38cccae871b94a9db82b5017c2c497c1)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:30.725142 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18230fb-d675-462f-bc27-7acafd91b68d admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:30.772723 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-b18230fb-d675-462f-bc27-7acafd91b68d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38cccae871b94a9db82b5017c2c497c1', 'request_id': 'req-b18230fb-d675-462f-bc27-7acafd91b68d'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:30.779679 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-b18230fb-d675-462f-bc27-7acafd91b68d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38cccae871b94a9db82b5017c2c497c1', 'request_id': 'req-b18230fb-d675-462f-bc27-7acafd91b68d'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:30.788197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-b18230fb-d675-462f-bc27-7acafd91b68d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38cccae871b94a9db82b5017c2c497c1', 'request_id': 'req-b18230fb-d675-462f-bc27-7acafd91b68d'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:30.803576 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-b18230fb-d675-462f-bc27-7acafd91b68d admin admin] The token cache is being invalidate because project 38cccae871b94a9db82b5017c2c497c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:40:30.803867 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-b18230fb-d675-462f-bc27-7acafd91b68d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:30.804663 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-b18230fb-d675-462f-bc27-7acafd91b68d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:30.807461 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1361/2696] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:40:30 2022] DELETE /identity/v3/projects/38cccae871b94a9db82b5017c2c497c1 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:30.812803 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739510d-5768-450c-a9cd-d651972c9d4a None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:30.813086 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739510d-5768-450c-a9cd-d651972c9d4a None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:30.813361 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739510d-5768-450c-a9cd-d651972c9d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:30.832798 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-e739510d-5768-450c-a9cd-d651972c9d4a None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:30.834042 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-e739510d-5768-450c-a9cd-d651972c9d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:30.870014 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:30.896433 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1336/2697] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:30.905010 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a49d1c-badf-429a-954b-7a3453f4c96d None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:30.905295 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a49d1c-badf-429a-954b-7a3453f4c96d None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:30.905569 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a49d1c-badf-429a-954b-7a3453f4c96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:30.925928 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-12a49d1c-badf-429a-954b-7a3453f4c96d None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:30.927124 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-12a49d1c-badf-429a-954b-7a3453f4c96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:30.935212 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1362/2698] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:30.939306 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e739510d-5768-450c-a9cd-d651972c9d4a None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:30.949420 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e739510d-5768-450c-a9cd-d651972c9d4a None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:30.956116 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452bce9-7e9b-4318-9ce1-00da1dc1843e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:30.957153 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452bce9-7e9b-4318-9ce1-00da1dc1843e admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:30.957657 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452bce9-7e9b-4318-9ce1-00da1dc1843e admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:30.958202 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452bce9-7e9b-4318-9ce1-00da1dc1843e admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:30.959296 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8452bce9-7e9b-4318-9ce1-00da1dc1843e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:40:30.959797 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8452bce9-7e9b-4318-9ce1-00da1dc1843e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:30.962624 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8452bce9-7e9b-4318-9ce1-00da1dc1843e admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:30.980854 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1337/2699] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:40:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:40:30.981769 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a49d1c-badf-429a-954b-7a3453f4c96d None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:30.991602 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a49d1c-badf-429a-954b-7a3453f4c96d None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:30.996468 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf3218-1307-436c-b7bd-c301a3988d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:30.997071 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf3218-1307-436c-b7bd-c301a3988d34 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:30.997386 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf3218-1307-436c-b7bd-c301a3988d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:30.997636 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf3218-1307-436c-b7bd-c301a3988d34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:30.998308 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf3218-1307-436c-b7bd-c301a3988d34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:31.000228 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf3218-1307-436c-b7bd-c301a3988d34 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:31.034823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1363/2700] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:40:30 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:40:31.036333 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452bce9-7e9b-4318-9ce1-00da1dc1843e admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:31.047847 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452bce9-7e9b-4318-9ce1-00da1dc1843e admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:31.112273 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7a545-8c5a-45c7-b684-eceec6a823bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:31.113256 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7a545-8c5a-45c7-b684-eceec6a823bd admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:31.113757 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7a545-8c5a-45c7-b684-eceec6a823bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:31.114309 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7a545-8c5a-45c7-b684-eceec6a823bd admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:31.115115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b7a545-8c5a-45c7-b684-eceec6a823bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:31.117327 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b7a545-8c5a-45c7-b684-eceec6a823bd admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:31.141847 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1338/2701] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:40:31 2022] POST /identity/v3/users => generated 284 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:40:31.145979 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf3218-1307-436c-b7bd-c301a3988d34 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:31.159401 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf3218-1307-436c-b7bd-c301a3988d34 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:31.166151 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3022043-6b29-403d-b03a-38963ba89d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:31.168291 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3022043-6b29-403d-b03a-38963ba89d71 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:31.168877 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3022043-6b29-403d-b03a-38963ba89d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:31.169157 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3022043-6b29-403d-b03a-38963ba89d71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:31.169795 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3022043-6b29-403d-b03a-38963ba89d71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:31.171462 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3022043-6b29-403d-b03a-38963ba89d71 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:31.184757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1364/2702] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:40:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:40:31.185900 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7a545-8c5a-45c7-b684-eceec6a823bd admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:31.196109 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7a545-8c5a-45c7-b684-eceec6a823bd admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:31.201253 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26392e93-13a6-43bc-b9ed-d6d4861f5655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:31.201911 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26392e93-13a6-43bc-b9ed-d6d4861f5655 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:31.202281 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26392e93-13a6-43bc-b9ed-d6d4861f5655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:31.202611 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26392e93-13a6-43bc-b9ed-d6d4861f5655 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:31.203305 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26392e93-13a6-43bc-b9ed-d6d4861f5655 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:31.205016 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26392e93-13a6-43bc-b9ed-d6d4861f5655 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:31.215945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1339/2703] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:40:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:40:31.220247 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3022043-6b29-403d-b03a-38963ba89d71 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:31.230552 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3022043-6b29-403d-b03a-38963ba89d71 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:31.235506 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f89d7dd-39ad-46b4-a29f-cc4bd14aed89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:31.236191 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89d7dd-39ad-46b4-a29f-cc4bd14aed89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:31.236476 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89d7dd-39ad-46b4-a29f-cc4bd14aed89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:31.236756 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89d7dd-39ad-46b4-a29f-cc4bd14aed89 admin admin] PATH_INFO: `/v3/projects/b68a04eb4ca34baa8c410a763998aba7/users/3859bace89eb45fbb4f37296afc2866c/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:31.255148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f89d7dd-39ad-46b4-a29f-cc4bd14aed89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b68a04eb4ca34baa8c410a763998aba7, user_id=3859bace89eb45fbb4f37296afc2866c, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:31.258245 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f89d7dd-39ad-46b4-a29f-cc4bd14aed89 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:31.273965 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1365/2704] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:40:31 2022] PUT /identity/v3/projects/b68a04eb4ca34baa8c410a763998aba7/users/3859bace89eb45fbb4f37296afc2866c/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:31.286140 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8466d-c281-4f55-883d-bca8d1539f9d None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:31.286660 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8466d-c281-4f55-883d-bca8d1539f9d None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:31.286799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8466d-c281-4f55-883d-bca8d1539f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:31.316547 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-a5c8466d-c281-4f55-883d-bca8d1539f9d None None] MFA Rules not processed for user `3859bace89eb45fbb4f37296afc2866c`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:31.317922 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-a5c8466d-c281-4f55-883d-bca8d1539f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:31.395939 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1340/2705] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:31.397000 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f89d7dd-39ad-46b4-a29f-cc4bd14aed89 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:31.429261 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:31.465420 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f89d7dd-39ad-46b4-a29f-cc4bd14aed89 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:31.470892 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b351b-60de-43e3-a756-4183d90b7e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:31.471567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b351b-60de-43e3-a756-4183d90b7e55 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:31.471866 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b351b-60de-43e3-a756-4183d90b7e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:31.472140 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b351b-60de-43e3-a756-4183d90b7e55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:31.472840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376b351b-60de-43e3-a756-4183d90b7e55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:31.474989 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376b351b-60de-43e3-a756-4183d90b7e55 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:31.506798 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1366/2706] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:40:31 2022] POST /identity/v3/projects => generated 367 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:40:31.511092 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8466d-c281-4f55-883d-bca8d1539f9d None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:31.522136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8466d-c281-4f55-883d-bca8d1539f9d None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:31.563929 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:31.596527 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee880f8b-16d1-4552-941f-66a2aa5fd01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:31.598730 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee880f8b-16d1-4552-941f-66a2aa5fd01b admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:31.598730 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee880f8b-16d1-4552-941f-66a2aa5fd01b admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:31.598730 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee880f8b-16d1-4552-941f-66a2aa5fd01b admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:31.598730 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee880f8b-16d1-4552-941f-66a2aa5fd01b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:31.600639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee880f8b-16d1-4552-941f-66a2aa5fd01b admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:31.623329 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1341/2707] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:40:31 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:40:31.627482 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b351b-60de-43e3-a756-4183d90b7e55 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:31.638553 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b351b-60de-43e3-a756-4183d90b7e55 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:31.643230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b0bb9-b371-4079-8bba-fd8790d7a2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:31.643821 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b0bb9-b371-4079-8bba-fd8790d7a2b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:31.644104 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b0bb9-b371-4079-8bba-fd8790d7a2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:31.644380 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b0bb9-b371-4079-8bba-fd8790d7a2b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:31.645051 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b0bb9-b371-4079-8bba-fd8790d7a2b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:31.646676 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b0bb9-b371-4079-8bba-fd8790d7a2b6 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:31.658266 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1367/2708] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:40:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:40:31.662479 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee880f8b-16d1-4552-941f-66a2aa5fd01b admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:31.672544 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee880f8b-16d1-4552-941f-66a2aa5fd01b admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:31.677149 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfda49d-5d29-4c8a-83ec-0b4607b6df98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:31.677800 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfda49d-5d29-4c8a-83ec-0b4607b6df98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:31.678110 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfda49d-5d29-4c8a-83ec-0b4607b6df98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:31.678387 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfda49d-5d29-4c8a-83ec-0b4607b6df98 admin admin] PATH_INFO: `/v3/projects/02666fe1bd7d4ae7a59f6148ff5740c2/users/bbff6ce4c7324748abae07d4947988d1/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:31.693666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfda49d-5d29-4c8a-83ec-0b4607b6df98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02666fe1bd7d4ae7a59f6148ff5740c2, user_id=bbff6ce4c7324748abae07d4947988d1, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:31.696293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfda49d-5d29-4c8a-83ec-0b4607b6df98 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:31.710821 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1342/2709] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:40:31 2022] PUT /identity/v3/projects/02666fe1bd7d4ae7a59f6148ff5740c2/users/bbff6ce4c7324748abae07d4947988d1/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:31.715558 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b0bb9-b371-4079-8bba-fd8790d7a2b6 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:31.726209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b0bb9-b371-4079-8bba-fd8790d7a2b6 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:31.785848 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a6023-537b-4f58-ab81-a6db41968a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:31.786497 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a6023-537b-4f58-ab81-a6db41968a34 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:31.786779 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a6023-537b-4f58-ab81-a6db41968a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:31.787119 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a6023-537b-4f58-ab81-a6db41968a34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:31.787689 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a6023-537b-4f58-ab81-a6db41968a34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:31.789420 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a6023-537b-4f58-ab81-a6db41968a34 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:31.804529 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1368/2710] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:40:31 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:40:31.808783 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfda49d-5d29-4c8a-83ec-0b4607b6df98 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:31.821214 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfda49d-5d29-4c8a-83ec-0b4607b6df98 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:31.826624 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3d3e0f-7c93-4fa1-a73c-c0e5d1e57b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:31.827290 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d3e0f-7c93-4fa1-a73c-c0e5d1e57b54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:31.827570 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d3e0f-7c93-4fa1-a73c-c0e5d1e57b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:31.827847 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d3e0f-7c93-4fa1-a73c-c0e5d1e57b54 admin admin] PATH_INFO: `/v3/projects/02666fe1bd7d4ae7a59f6148ff5740c2/users/bbff6ce4c7324748abae07d4947988d1/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:31.829779 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3d3e0f-7c93-4fa1-a73c-c0e5d1e57b54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02666fe1bd7d4ae7a59f6148ff5740c2, user_id=bbff6ce4c7324748abae07d4947988d1, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:31.832428 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3d3e0f-7c93-4fa1-a73c-c0e5d1e57b54 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:31.843768 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1343/2711] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:40:31 2022] PUT /identity/v3/projects/02666fe1bd7d4ae7a59f6148ff5740c2/users/bbff6ce4c7324748abae07d4947988d1/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:31.854103 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a076120-5eef-4dee-b002-30f2105030b1 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:31.854397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a076120-5eef-4dee-b002-30f2105030b1 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:31.854676 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a076120-5eef-4dee-b002-30f2105030b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:31.885383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-0a076120-5eef-4dee-b002-30f2105030b1 None None] MFA Rules not processed for user `bbff6ce4c7324748abae07d4947988d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:31.886641 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-0a076120-5eef-4dee-b002-30f2105030b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:31.957040 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1369/2712] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:31 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:31.968884 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3d3e0f-7c93-4fa1-a73c-c0e5d1e57b54 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:31.970709 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-2a3d3e0f-7c93-4fa1-a73c-c0e5d1e57b54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:32.040069 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3d3e0f-7c93-4fa1-a73c-c0e5d1e57b54 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:32.044408 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec8f77-a0f5-482b-afe8-4e643a93a8a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:32.045042 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec8f77-a0f5-482b-afe8-4e643a93a8a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:32.045315 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec8f77-a0f5-482b-afe8-4e643a93a8a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:32.046057 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec8f77-a0f5-482b-afe8-4e643a93a8a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:32.056485 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec8f77-a0f5-482b-afe8-4e643a93a8a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:32.059511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec8f77-a0f5-482b-afe8-4e643a93a8a2 service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:32.069494 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1344/2713] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:40:31 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:40:33.491311 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a076120-5eef-4dee-b002-30f2105030b1 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:33.519752 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:33.564964 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a076120-5eef-4dee-b002-30f2105030b1 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:33.569986 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c541a-e381-407e-b1a7-e6f1db5d061f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:33.570642 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c541a-e381-407e-b1a7-e6f1db5d061f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:33.570917 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c541a-e381-407e-b1a7-e6f1db5d061f admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:33.571188 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c541a-e381-407e-b1a7-e6f1db5d061f admin admin] PATH_INFO: `/v3/users/3859bace89eb45fbb4f37296afc2866c` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:33.572834 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c541a-e381-407e-b1a7-e6f1db5d061f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3859bace89eb45fbb4f37296afc2866c)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:33.574764 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c541a-e381-407e-b1a7-e6f1db5d061f admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:33.634556 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-480c541a-e381-407e-b1a7-e6f1db5d061f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3859bace89eb45fbb4f37296afc2866c'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:33.635676 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-480c541a-e381-407e-b1a7-e6f1db5d061f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3859bace89eb45fbb4f37296afc2866c'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:33.645165 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-480c541a-e381-407e-b1a7-e6f1db5d061f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3859bace89eb45fbb4f37296afc2866c'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:33.658074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-480c541a-e381-407e-b1a7-e6f1db5d061f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3859bace89eb45fbb4f37296afc2866c'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:33.659156 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-480c541a-e381-407e-b1a7-e6f1db5d061f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3859bace89eb45fbb4f37296afc2866c'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:33.673323 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1370/2714] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:40:33 2022] DELETE /identity/v3/users/3859bace89eb45fbb4f37296afc2866c => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:33.677795 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f412f6-ab82-4499-8102-e1a506cd74b8 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:33.678285 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f412f6-ab82-4499-8102-e1a506cd74b8 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:33.678383 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f412f6-ab82-4499-8102-e1a506cd74b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:33.698825 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-81f412f6-ab82-4499-8102-e1a506cd74b8 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:33.700063 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-81f412f6-ab82-4499-8102-e1a506cd74b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:33.789800 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1345/2715] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:40:33.803946 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c541a-e381-407e-b1a7-e6f1db5d061f admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:33.806074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-480c541a-e381-407e-b1a7-e6f1db5d061f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:33.898387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c541a-e381-407e-b1a7-e6f1db5d061f admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:33.903561 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994d0ff-fe1a-4a83-ae2f-5a261b2a2f30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:33.904404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994d0ff-fe1a-4a83-ae2f-5a261b2a2f30 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:33.904833 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994d0ff-fe1a-4a83-ae2f-5a261b2a2f30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:33.905317 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994d0ff-fe1a-4a83-ae2f-5a261b2a2f30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:33.918643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4994d0ff-fe1a-4a83-ae2f-5a261b2a2f30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:33.920389 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4994d0ff-fe1a-4a83-ae2f-5a261b2a2f30 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:33.932365 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1371/2716] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:40:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:40:34.213972 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f412f6-ab82-4499-8102-e1a506cd74b8 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:34.215645 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-81f412f6-ab82-4499-8102-e1a506cd74b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:34.228061 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f412f6-ab82-4499-8102-e1a506cd74b8 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:34.234335 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31122cec-f58c-44f3-8319-8f4797fc6f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:34.235157 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31122cec-f58c-44f3-8319-8f4797fc6f51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:34.235436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31122cec-f58c-44f3-8319-8f4797fc6f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:34.235705 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31122cec-f58c-44f3-8319-8f4797fc6f51 admin admin] PATH_INFO: `/v3/projects/b68a04eb4ca34baa8c410a763998aba7` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:34.236874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31122cec-f58c-44f3-8319-8f4797fc6f51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b68a04eb4ca34baa8c410a763998aba7)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:34.238797 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31122cec-f58c-44f3-8319-8f4797fc6f51 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:34.274410 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:34.293780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-31122cec-f58c-44f3-8319-8f4797fc6f51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b68a04eb4ca34baa8c410a763998aba7', 'request_id': 'req-31122cec-f58c-44f3-8319-8f4797fc6f51'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:34.300137 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-31122cec-f58c-44f3-8319-8f4797fc6f51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b68a04eb4ca34baa8c410a763998aba7', 'request_id': 'req-31122cec-f58c-44f3-8319-8f4797fc6f51'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:34.306125 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-31122cec-f58c-44f3-8319-8f4797fc6f51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b68a04eb4ca34baa8c410a763998aba7', 'request_id': 'req-31122cec-f58c-44f3-8319-8f4797fc6f51'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:34.318375 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-31122cec-f58c-44f3-8319-8f4797fc6f51 admin admin] The token cache is being invalidate because project b68a04eb4ca34baa8c410a763998aba7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:40:34.318661 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-31122cec-f58c-44f3-8319-8f4797fc6f51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:34.319292 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-31122cec-f58c-44f3-8319-8f4797fc6f51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:34.320768 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1346/2717] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:40:34 2022] DELETE /identity/v3/projects/b68a04eb4ca34baa8c410a763998aba7 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:34.325305 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994d0ff-fe1a-4a83-ae2f-5a261b2a2f30 service neutron] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:34.334892 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994d0ff-fe1a-4a83-ae2f-5a261b2a2f30 service neutron] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:34.379530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:34.395015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499406d6-72b4-479a-aff4-ddf7cbecd090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:34.395644 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499406d6-72b4-479a-aff4-ddf7cbecd090 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:34.395923 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499406d6-72b4-479a-aff4-ddf7cbecd090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:34.396192 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499406d6-72b4-479a-aff4-ddf7cbecd090 admin admin] PATH_INFO: `/v3/users/bbff6ce4c7324748abae07d4947988d1` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:34.397301 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499406d6-72b4-479a-aff4-ddf7cbecd090 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbff6ce4c7324748abae07d4947988d1)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:34.399235 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499406d6-72b4-479a-aff4-ddf7cbecd090 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:34.444404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-499406d6-72b4-479a-aff4-ddf7cbecd090 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbff6ce4c7324748abae07d4947988d1'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:34.445045 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-499406d6-72b4-479a-aff4-ddf7cbecd090 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbff6ce4c7324748abae07d4947988d1'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:34.452769 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-499406d6-72b4-479a-aff4-ddf7cbecd090 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbff6ce4c7324748abae07d4947988d1'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:34.464903 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-499406d6-72b4-479a-aff4-ddf7cbecd090 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbff6ce4c7324748abae07d4947988d1'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:34.465568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-499406d6-72b4-479a-aff4-ddf7cbecd090 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbff6ce4c7324748abae07d4947988d1'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:34.477952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1372/2718] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:40:34 2022] DELETE /identity/v3/users/bbff6ce4c7324748abae07d4947988d1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:34.686911 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31122cec-f58c-44f3-8319-8f4797fc6f51 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:34.688450 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-31122cec-f58c-44f3-8319-8f4797fc6f51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:34.702227 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31122cec-f58c-44f3-8319-8f4797fc6f51 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:34.765354 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8aca7-2c3d-4ce4-a060-f77fcb9271ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:34.765995 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8aca7-2c3d-4ce4-a060-f77fcb9271ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:34.766314 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8aca7-2c3d-4ce4-a060-f77fcb9271ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:34.766589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8aca7-2c3d-4ce4-a060-f77fcb9271ec admin admin] PATH_INFO: `/v3/users/5c5ba905177e4f71908396ba3689f47d` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:34.767750 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc8aca7-2c3d-4ce4-a060-f77fcb9271ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c5ba905177e4f71908396ba3689f47d)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:34.769691 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc8aca7-2c3d-4ce4-a060-f77fcb9271ec admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:34.825474 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6dc8aca7-2c3d-4ce4-a060-f77fcb9271ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c5ba905177e4f71908396ba3689f47d'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:34.826299 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6dc8aca7-2c3d-4ce4-a060-f77fcb9271ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c5ba905177e4f71908396ba3689f47d'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:34.826906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6dc8aca7-2c3d-4ce4-a060-f77fcb9271ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c5ba905177e4f71908396ba3689f47d'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:34.835072 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6dc8aca7-2c3d-4ce4-a060-f77fcb9271ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c5ba905177e4f71908396ba3689f47d'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:34.848627 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6dc8aca7-2c3d-4ce4-a060-f77fcb9271ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c5ba905177e4f71908396ba3689f47d'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:34.864249 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1347/2719] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:40:34 2022] DELETE /identity/v3/users/5c5ba905177e4f71908396ba3689f47d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:34.870874 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0602154-ee66-49a0-af6d-e8c73e14c580 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:34.871625 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0602154-ee66-49a0-af6d-e8c73e14c580 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:34.872124 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0602154-ee66-49a0-af6d-e8c73e14c580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:34.928745 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-e0602154-ee66-49a0-af6d-e8c73e14c580 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:34.930554 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-e0602154-ee66-49a0-af6d-e8c73e14c580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:35.065503 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1373/2720] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:35.078961 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8aca7-2c3d-4ce4-a060-f77fcb9271ec admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:35.081426 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-6dc8aca7-2c3d-4ce4-a060-f77fcb9271ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:35.140440 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:35.174003 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8aca7-2c3d-4ce4-a060-f77fcb9271ec admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:35.180486 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6d8298-f6b5-4c6d-82bb-a89689e523e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:35.182155 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d8298-f6b5-4c6d-82bb-a89689e523e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:35.183175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d8298-f6b5-4c6d-82bb-a89689e523e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:35.183813 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d8298-f6b5-4c6d-82bb-a89689e523e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:35.190191 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0602154-ee66-49a0-af6d-e8c73e14c580 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:35.191604 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-e0602154-ee66-49a0-af6d-e8c73e14c580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:35.196967 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6d8298-f6b5-4c6d-82bb-a89689e523e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:35.198532 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6d8298-f6b5-4c6d-82bb-a89689e523e8 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:35.210675 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1348/2721] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:40:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:40:35.236232 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:35.240851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0602154-ee66-49a0-af6d-e8c73e14c580 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:35.248050 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12dc65-cf70-442b-a002-8f782814bf42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:35.249326 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12dc65-cf70-442b-a002-8f782814bf42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:35.249684 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12dc65-cf70-442b-a002-8f782814bf42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:35.249962 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12dc65-cf70-442b-a002-8f782814bf42 admin admin] PATH_INFO: `/v3/projects/02666fe1bd7d4ae7a59f6148ff5740c2` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:35.252046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff12dc65-cf70-442b-a002-8f782814bf42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02666fe1bd7d4ae7a59f6148ff5740c2)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:35.254803 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff12dc65-cf70-442b-a002-8f782814bf42 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:35.322234 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-ff12dc65-cf70-442b-a002-8f782814bf42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02666fe1bd7d4ae7a59f6148ff5740c2', 'request_id': 'req-ff12dc65-cf70-442b-a002-8f782814bf42'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:35.329349 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-ff12dc65-cf70-442b-a002-8f782814bf42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02666fe1bd7d4ae7a59f6148ff5740c2', 'request_id': 'req-ff12dc65-cf70-442b-a002-8f782814bf42'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:35.339097 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-ff12dc65-cf70-442b-a002-8f782814bf42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02666fe1bd7d4ae7a59f6148ff5740c2', 'request_id': 'req-ff12dc65-cf70-442b-a002-8f782814bf42'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:35.350920 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-ff12dc65-cf70-442b-a002-8f782814bf42 admin admin] The token cache is being invalidate because project 02666fe1bd7d4ae7a59f6148ff5740c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:40:35.351255 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-ff12dc65-cf70-442b-a002-8f782814bf42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:35.351860 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-ff12dc65-cf70-442b-a002-8f782814bf42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:35.353358 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1374/2722] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:40:35 2022] DELETE /identity/v3/projects/02666fe1bd7d4ae7a59f6148ff5740c2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:40:35.358978 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d9e23-942b-4190-b273-812bd7988251 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:35.359264 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d9e23-942b-4190-b273-812bd7988251 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:35.359539 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d9e23-942b-4190-b273-812bd7988251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:35.418727 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-c87d9e23-942b-4190-b273-812bd7988251 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:35.420976 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-c87d9e23-942b-4190-b273-812bd7988251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:35.504443 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1349/2723] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:40:35.513756 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5882521-e476-4605-ab96-3a42e216ba74 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:35.514262 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5882521-e476-4605-ab96-3a42e216ba74 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:35.514701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5882521-e476-4605-ab96-3a42e216ba74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:35.535573 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-e5882521-e476-4605-ab96-3a42e216ba74 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:35.536906 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-e5882521-e476-4605-ab96-3a42e216ba74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:35.545120 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1375/2724] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:35.549293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d9e23-942b-4190-b273-812bd7988251 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:35.560992 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d9e23-942b-4190-b273-812bd7988251 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:35.566009 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79175ed3-214e-44d9-b9e1-53a39391b707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:35.566645 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79175ed3-214e-44d9-b9e1-53a39391b707 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:35.566928 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79175ed3-214e-44d9-b9e1-53a39391b707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:35.567205 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79175ed3-214e-44d9-b9e1-53a39391b707 admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:35.567866 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79175ed3-214e-44d9-b9e1-53a39391b707 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:40:35.568175 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79175ed3-214e-44d9-b9e1-53a39391b707 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:35.569806 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79175ed3-214e-44d9-b9e1-53a39391b707 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:35.586013 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1350/2725] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:40:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:40:35.586201 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5882521-e476-4605-ab96-3a42e216ba74 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:35.587146 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-e5882521-e476-4605-ab96-3a42e216ba74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:35.591075 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79175ed3-214e-44d9-b9e1-53a39391b707 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:35.600434 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5882521-e476-4605-ab96-3a42e216ba74 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:35.607997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b144b3e8-a750-4cb6-b592-f12396469589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:35.607997 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144b3e8-a750-4cb6-b592-f12396469589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:35.608526 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144b3e8-a750-4cb6-b592-f12396469589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:35.608962 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144b3e8-a750-4cb6-b592-f12396469589 admin admin] PATH_INFO: `/v3/projects/0ee33b2a73604a70bc5e0bd14d1ff109` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:35.610541 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b144b3e8-a750-4cb6-b592-f12396469589 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ee33b2a73604a70bc5e0bd14d1ff109)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:35.614286 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79175ed3-214e-44d9-b9e1-53a39391b707 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:35.614519 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b144b3e8-a750-4cb6-b592-f12396469589 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:35.621197 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa1104-04c4-4719-9d82-e1f7b98d59c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:35.621864 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa1104-04c4-4719-9d82-e1f7b98d59c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:35.622246 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa1104-04c4-4719-9d82-e1f7b98d59c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:35.622453 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa1104-04c4-4719-9d82-e1f7b98d59c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:35.623250 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aa1104-04c4-4719-9d82-e1f7b98d59c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:35.625313 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aa1104-04c4-4719-9d82-e1f7b98d59c4 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:35.655162 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1351/2726] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:40:35 2022] POST /identity/v3/projects => generated 383 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:40:35.659327 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa1104-04c4-4719-9d82-e1f7b98d59c4 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:35.667383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-b144b3e8-a750-4cb6-b592-f12396469589 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ee33b2a73604a70bc5e0bd14d1ff109', 'request_id': 'req-b144b3e8-a750-4cb6-b592-f12396469589'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:35.673688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-b144b3e8-a750-4cb6-b592-f12396469589 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ee33b2a73604a70bc5e0bd14d1ff109', 'request_id': 'req-b144b3e8-a750-4cb6-b592-f12396469589'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:35.681858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-b144b3e8-a750-4cb6-b592-f12396469589 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ee33b2a73604a70bc5e0bd14d1ff109', 'request_id': 'req-b144b3e8-a750-4cb6-b592-f12396469589'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:35.693055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-b144b3e8-a750-4cb6-b592-f12396469589 admin admin] The token cache is being invalidate because project 0ee33b2a73604a70bc5e0bd14d1ff109 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:40:35.693345 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-b144b3e8-a750-4cb6-b592-f12396469589 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:35.693980 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-b144b3e8-a750-4cb6-b592-f12396469589 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:35.695521 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1376/2727] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:40:35 2022] DELETE /identity/v3/projects/0ee33b2a73604a70bc5e0bd14d1ff109 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:40:35.725434 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa1104-04c4-4719-9d82-e1f7b98d59c4 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:35.730383 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7d846-5f43-45a4-8a23-3a716b21f073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:35.731018 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7d846-5f43-45a4-8a23-3a716b21f073 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:35.731297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7d846-5f43-45a4-8a23-3a716b21f073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:35.731569 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7d846-5f43-45a4-8a23-3a716b21f073 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:35.732481 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7d846-5f43-45a4-8a23-3a716b21f073 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:35.734475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7d846-5f43-45a4-8a23-3a716b21f073 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:35.746693 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:35.759211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1352/2728] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:40:35 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:40:35.764208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b144b3e8-a750-4cb6-b592-f12396469589 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:35.765937 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-b144b3e8-a750-4cb6-b592-f12396469589 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:35.844133 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b144b3e8-a750-4cb6-b592-f12396469589 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:35.849169 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695829a-6eb6-4bcc-a87e-251028d403f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:35.849807 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695829a-6eb6-4bcc-a87e-251028d403f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:35.850184 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695829a-6eb6-4bcc-a87e-251028d403f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:35.850458 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695829a-6eb6-4bcc-a87e-251028d403f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:35.851118 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695829a-6eb6-4bcc-a87e-251028d403f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:35.852749 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695829a-6eb6-4bcc-a87e-251028d403f6 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:35.860115 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:35.866435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1377/2729] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:40:35 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:40:35.871961 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7d846-5f43-45a4-8a23-3a716b21f073 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:35.882442 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7d846-5f43-45a4-8a23-3a716b21f073 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:35.888943 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430bf1d0-3fb4-4e2b-8d7b-e52a15e61130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:35.889560 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430bf1d0-3fb4-4e2b-8d7b-e52a15e61130 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:35.889833 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430bf1d0-3fb4-4e2b-8d7b-e52a15e61130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:35.890128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430bf1d0-3fb4-4e2b-8d7b-e52a15e61130 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:35.890785 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430bf1d0-3fb4-4e2b-8d7b-e52a15e61130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:35.892413 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430bf1d0-3fb4-4e2b-8d7b-e52a15e61130 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:35.907509 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1353/2730] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:40:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:40:35.908779 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695829a-6eb6-4bcc-a87e-251028d403f6 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:35.927662 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695829a-6eb6-4bcc-a87e-251028d403f6 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:35.933741 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f06faf-ade4-483f-8aa7-fc2b5f8fbd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:35.934004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f06faf-ade4-483f-8aa7-fc2b5f8fbd04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:35.934283 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f06faf-ade4-483f-8aa7-fc2b5f8fbd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:35.934580 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f06faf-ade4-483f-8aa7-fc2b5f8fbd04 admin admin] PATH_INFO: `/v3/projects/a798ba18b97d4cd7a5ed24223870c0e7/users/d951450c1004415bade76d83e7d220e2/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:35.951077 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f06faf-ade4-483f-8aa7-fc2b5f8fbd04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a798ba18b97d4cd7a5ed24223870c0e7, user_id=d951450c1004415bade76d83e7d220e2, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:35.953755 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f06faf-ade4-483f-8aa7-fc2b5f8fbd04 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:35.967778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1378/2731] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:40:35 2022] PUT /identity/v3/projects/a798ba18b97d4cd7a5ed24223870c0e7/users/d951450c1004415bade76d83e7d220e2/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:40:35.979660 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cee00-a083-4413-b3c5-d4dab640b518 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:35.979937 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cee00-a083-4413-b3c5-d4dab640b518 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:35.980231 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cee00-a083-4413-b3c5-d4dab640b518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:36.008003 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-d04cee00-a083-4413-b3c5-d4dab640b518 None None] MFA Rules not processed for user `d951450c1004415bade76d83e7d220e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:36.009223 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-d04cee00-a083-4413-b3c5-d4dab640b518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:36.071901 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1354/2732] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:36.076612 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f06faf-ade4-483f-8aa7-fc2b5f8fbd04 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:36.138580 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f06faf-ade4-483f-8aa7-fc2b5f8fbd04 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:36.143139 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439f7b2-862b-442b-81c3-e25bc7cb4db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:36.143737 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439f7b2-862b-442b-81c3-e25bc7cb4db7 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:36.144016 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439f7b2-862b-442b-81c3-e25bc7cb4db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:36.144290 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439f7b2-862b-442b-81c3-e25bc7cb4db7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:36.144924 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5439f7b2-862b-442b-81c3-e25bc7cb4db7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:36.146856 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5439f7b2-862b-442b-81c3-e25bc7cb4db7 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:36.174413 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1379/2733] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:40:36 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:40:36.178577 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04cee00-a083-4413-b3c5-d4dab640b518 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:36.189101 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04cee00-a083-4413-b3c5-d4dab640b518 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:36.246102 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf56d67-dc62-4dd6-aa51-40cbcd9597eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:36.246713 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf56d67-dc62-4dd6-aa51-40cbcd9597eb admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:36.246991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf56d67-dc62-4dd6-aa51-40cbcd9597eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:36.247262 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf56d67-dc62-4dd6-aa51-40cbcd9597eb admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:36.247899 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf56d67-dc62-4dd6-aa51-40cbcd9597eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:36.249783 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf56d67-dc62-4dd6-aa51-40cbcd9597eb admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:36.272180 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1355/2734] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:40:36 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:40:36.276574 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439f7b2-862b-442b-81c3-e25bc7cb4db7 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:36.289302 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439f7b2-862b-442b-81c3-e25bc7cb4db7 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:36.295460 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415af77-ef63-4764-809e-53aeb51e9b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:36.296075 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415af77-ef63-4764-809e-53aeb51e9b68 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:36.296482 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415af77-ef63-4764-809e-53aeb51e9b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:36.296897 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415af77-ef63-4764-809e-53aeb51e9b68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:36.297779 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415af77-ef63-4764-809e-53aeb51e9b68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:36.299604 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415af77-ef63-4764-809e-53aeb51e9b68 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:36.312939 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1380/2735] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:40:36.317444 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf56d67-dc62-4dd6-aa51-40cbcd9597eb admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:36.333499 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf56d67-dc62-4dd6-aa51-40cbcd9597eb admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:36.338994 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aec475-0dad-4958-965c-3066a9d7ecc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:36.339795 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aec475-0dad-4958-965c-3066a9d7ecc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:36.340157 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aec475-0dad-4958-965c-3066a9d7ecc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:36.340719 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aec475-0dad-4958-965c-3066a9d7ecc9 admin admin] PATH_INFO: `/v3/projects/1c48d105fbcc4bd48caebe7742abe2ae/users/8a9217595eeb4931ad35b4596a2281b9/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:36.359919 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aec475-0dad-4958-965c-3066a9d7ecc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c48d105fbcc4bd48caebe7742abe2ae, user_id=8a9217595eeb4931ad35b4596a2281b9, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:36.362748 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aec475-0dad-4958-965c-3066a9d7ecc9 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:36.376589 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1356/2736] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:40:36 2022] PUT /identity/v3/projects/1c48d105fbcc4bd48caebe7742abe2ae/users/8a9217595eeb4931ad35b4596a2281b9/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:36.380517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415af77-ef63-4764-809e-53aeb51e9b68 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:36.391907 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415af77-ef63-4764-809e-53aeb51e9b68 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:36.448741 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc56afed-6438-4799-ac6e-3b041ac5999d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:36.449367 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56afed-6438-4799-ac6e-3b041ac5999d admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:36.449642 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56afed-6438-4799-ac6e-3b041ac5999d admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:36.449911 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56afed-6438-4799-ac6e-3b041ac5999d admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:36.450581 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc56afed-6438-4799-ac6e-3b041ac5999d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:36.452208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc56afed-6438-4799-ac6e-3b041ac5999d admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:36.461769 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1381/2737] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:40:36.466978 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aec475-0dad-4958-965c-3066a9d7ecc9 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:36.477018 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aec475-0dad-4958-965c-3066a9d7ecc9 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:36.481522 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b3fa6-0f6a-4782-a611-8d581cbe878f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:36.482461 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b3fa6-0f6a-4782-a611-8d581cbe878f admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:36.482740 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b3fa6-0f6a-4782-a611-8d581cbe878f admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:36.483008 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b3fa6-0f6a-4782-a611-8d581cbe878f admin admin] PATH_INFO: `/v3/projects/1c48d105fbcc4bd48caebe7742abe2ae/users/8a9217595eeb4931ad35b4596a2281b9/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:36.484873 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b3fa6-0f6a-4782-a611-8d581cbe878f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c48d105fbcc4bd48caebe7742abe2ae, user_id=8a9217595eeb4931ad35b4596a2281b9, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:36.487487 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b3fa6-0f6a-4782-a611-8d581cbe878f admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:36.497420 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1357/2738] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:40:36 2022] PUT /identity/v3/projects/1c48d105fbcc4bd48caebe7742abe2ae/users/8a9217595eeb4931ad35b4596a2281b9/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:36.507375 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ac7b8-f965-4709-b930-2dc7da54ebde None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:36.507663 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ac7b8-f965-4709-b930-2dc7da54ebde None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:36.507934 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ac7b8-f965-4709-b930-2dc7da54ebde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:36.537981 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-e53ac7b8-f965-4709-b930-2dc7da54ebde None None] MFA Rules not processed for user `8a9217595eeb4931ad35b4596a2281b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:36.539227 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-e53ac7b8-f965-4709-b930-2dc7da54ebde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:36.574095 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:36.613681 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1382/2739] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:36 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:40:36.625690 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b3fa6-0f6a-4782-a611-8d581cbe878f admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:36.627477 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-8d5b3fa6-0f6a-4782-a611-8d581cbe878f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:36.658257 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:36.704873 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b3fa6-0f6a-4782-a611-8d581cbe878f admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:36.708536 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6d912-926b-4cd1-95f4-cc5ca0137592 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:36.709131 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6d912-926b-4cd1-95f4-cc5ca0137592 service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:36.709408 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6d912-926b-4cd1-95f4-cc5ca0137592 service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:36.709671 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6d912-926b-4cd1-95f4-cc5ca0137592 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:36.765827 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6d912-926b-4cd1-95f4-cc5ca0137592 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:36.767338 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6d912-926b-4cd1-95f4-cc5ca0137592 service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:36.776248 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1358/2740] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:40:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:40:38.997558 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0faf7c-bb4f-4af7-88eb-52e90b91448f None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:38.998094 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0faf7c-bb4f-4af7-88eb-52e90b91448f None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:38.998204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0faf7c-bb4f-4af7-88eb-52e90b91448f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:39.030309 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-6d0faf7c-bb4f-4af7-88eb-52e90b91448f None None] MFA Rules not processed for user `9158557f1a2947e2815dc8d586d8b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:39.031547 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-6d0faf7c-bb4f-4af7-88eb-52e90b91448f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:39.093200 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1383/2741] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Sep 26 10:40:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:39.138504 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6d912-926b-4cd1-95f4-cc5ca0137592 service cinder] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:39.139886 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-d3f6d912-926b-4cd1-95f4-cc5ca0137592 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:39.207197 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6d912-926b-4cd1-95f4-cc5ca0137592 service cinder] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:39.212099 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57060e0e-7c79-40ff-ae74-27f618193311 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e6be34fc93a4551a83e5b8380188a6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:39.212681 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57060e0e-7c79-40ff-ae74-27f618193311 service swift] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:39.212959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57060e0e-7c79-40ff-ae74-27f618193311 service swift] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:39.213761 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57060e0e-7c79-40ff-ae74-27f618193311 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:39.221592 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57060e0e-7c79-40ff-ae74-27f618193311 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:39.223171 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57060e0e-7c79-40ff-ae74-27f618193311 service swift] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:39.231605 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1359/2742] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 10:40:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 10:40:39.747005 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f71f48-d250-4327-ae62-aaec74d9f722 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:39.747297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f71f48-d250-4327-ae62-aaec74d9f722 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:39.747625 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f71f48-d250-4327-ae62-aaec74d9f722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:39.770139 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-00f71f48-d250-4327-ae62-aaec74d9f722 None None] MFA Rules not processed for user `9158557f1a2947e2815dc8d586d8b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:39.771568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-00f71f48-d250-4327-ae62-aaec74d9f722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:39.779842 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1384/2743] 10.222.0.49 () {58 vars in 967 bytes} [Mon Sep 26 10:40:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:39.795769 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57060e0e-7c79-40ff-ae74-27f618193311 service swift] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:39.808203 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57060e0e-7c79-40ff-ae74-27f618193311 service swift] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:39.814645 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060dcb98-337b-42ca-8521-cc1b855e6d52 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e6be34fc93a4551a83e5b8380188a6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:39.815337 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060dcb98-337b-42ca-8521-cc1b855e6d52 service swift] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:39.815725 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060dcb98-337b-42ca-8521-cc1b855e6d52 service swift] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:39.816165 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060dcb98-337b-42ca-8521-cc1b855e6d52 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:39.825447 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060dcb98-337b-42ca-8521-cc1b855e6d52 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:39.827150 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060dcb98-337b-42ca-8521-cc1b855e6d52 service swift] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:39.836358 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1360/2744] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 10:40:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 10:40:40.100335 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c35199-0c90-4f8b-92a2-d992c82ac261 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:40.100807 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c35199-0c90-4f8b-92a2-d992c82ac261 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:40.100916 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c35199-0c90-4f8b-92a2-d992c82ac261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:40.120265 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-c9c35199-0c90-4f8b-92a2-d992c82ac261 None None] MFA Rules not processed for user `9158557f1a2947e2815dc8d586d8b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:40.121392 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-c9c35199-0c90-4f8b-92a2-d992c82ac261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:40.128423 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1385/2745] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Sep 26 10:40:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:40.140547 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060dcb98-337b-42ca-8521-cc1b855e6d52 service swift] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:40.151947 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060dcb98-337b-42ca-8521-cc1b855e6d52 service swift] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:40.156915 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee228904-69f9-4063-b560-1adf1c106ca3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e6be34fc93a4551a83e5b8380188a6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:40.157529 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee228904-69f9-4063-b560-1adf1c106ca3 service swift] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:40.157805 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee228904-69f9-4063-b560-1adf1c106ca3 service swift] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:40.158106 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee228904-69f9-4063-b560-1adf1c106ca3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:40.166251 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee228904-69f9-4063-b560-1adf1c106ca3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:40.167820 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee228904-69f9-4063-b560-1adf1c106ca3 service swift] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:40.176073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1361/2746] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 10:40:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 10:40:40.487840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e698a12-a418-4752-b88d-8a8bbac682fd None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:40.488127 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e698a12-a418-4752-b88d-8a8bbac682fd None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:40.488407 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e698a12-a418-4752-b88d-8a8bbac682fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:40.508858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-9e698a12-a418-4752-b88d-8a8bbac682fd None None] MFA Rules not processed for user `9158557f1a2947e2815dc8d586d8b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:40.510153 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-9e698a12-a418-4752-b88d-8a8bbac682fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:40.517802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1386/2747] 10.222.0.49 () {58 vars in 967 bytes} [Mon Sep 26 10:40:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:40.529760 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee228904-69f9-4063-b560-1adf1c106ca3 service swift] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:40.540537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee228904-69f9-4063-b560-1adf1c106ca3 service swift] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:40.545484 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cadcaa-f07a-46bf-a854-950404c777fd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e6be34fc93a4551a83e5b8380188a6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:40.546091 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cadcaa-f07a-46bf-a854-950404c777fd service swift] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:40.546368 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cadcaa-f07a-46bf-a854-950404c777fd service swift] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:40.546633 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cadcaa-f07a-46bf-a854-950404c777fd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:40.554629 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cadcaa-f07a-46bf-a854-950404c777fd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:40.556178 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cadcaa-f07a-46bf-a854-950404c777fd service swift] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:40.564588 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1362/2748] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Sep 26 10:40:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 26 10:40:41.034128 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e698a12-a418-4752-b88d-8a8bbac682fd None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:41.044350 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e698a12-a418-4752-b88d-8a8bbac682fd None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:41.047981 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba453b-773d-4754-acc7-9f8381d17f10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:41.048582 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba453b-773d-4754-acc7-9f8381d17f10 service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:41.048858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba453b-773d-4754-acc7-9f8381d17f10 service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:41.049125 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba453b-773d-4754-acc7-9f8381d17f10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:41.058742 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba453b-773d-4754-acc7-9f8381d17f10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:41.060156 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba453b-773d-4754-acc7-9f8381d17f10 service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:41.069683 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1387/2749] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:40:41 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:40:42.737326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cadcaa-f07a-46bf-a854-950404c777fd service swift] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:42.738825 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-48cadcaa-f07a-46bf-a854-950404c777fd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:42.756902 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:42.804761 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cadcaa-f07a-46bf-a854-950404c777fd service swift] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:42.809318 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51525a91-0fbc-4be4-8c95-5e24a53d90ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:42.809958 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51525a91-0fbc-4be4-8c95-5e24a53d90ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:42.810600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51525a91-0fbc-4be4-8c95-5e24a53d90ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:42.810600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51525a91-0fbc-4be4-8c95-5e24a53d90ea admin admin] PATH_INFO: `/v3/users/1f631f56726c4cab8bd7d61076208b18` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:42.811643 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51525a91-0fbc-4be4-8c95-5e24a53d90ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f631f56726c4cab8bd7d61076208b18)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:42.813517 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51525a91-0fbc-4be4-8c95-5e24a53d90ea admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:42.862948 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-51525a91-0fbc-4be4-8c95-5e24a53d90ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f631f56726c4cab8bd7d61076208b18'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:42.863614 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-51525a91-0fbc-4be4-8c95-5e24a53d90ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f631f56726c4cab8bd7d61076208b18'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:42.865278 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-51525a91-0fbc-4be4-8c95-5e24a53d90ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f631f56726c4cab8bd7d61076208b18'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:42.877016 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-51525a91-0fbc-4be4-8c95-5e24a53d90ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f631f56726c4cab8bd7d61076208b18'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:42.889098 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-51525a91-0fbc-4be4-8c95-5e24a53d90ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f631f56726c4cab8bd7d61076208b18'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:42.901814 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1363/2750] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:40:42 2022] DELETE /identity/v3/users/1f631f56726c4cab8bd7d61076208b18 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:42.906469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b850de-8c2d-400b-bc3a-867aa97cec08 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:42.906931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b850de-8c2d-400b-bc3a-867aa97cec08 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:42.907030 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b850de-8c2d-400b-bc3a-867aa97cec08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:42.926242 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-b3b850de-8c2d-400b-bc3a-867aa97cec08 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:42.927452 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-b3b850de-8c2d-400b-bc3a-867aa97cec08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:42.985907 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1388/2751] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:42.997378 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51525a91-0fbc-4be4-8c95-5e24a53d90ea admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:42.998918 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-51525a91-0fbc-4be4-8c95-5e24a53d90ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:43.055548 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51525a91-0fbc-4be4-8c95-5e24a53d90ea admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:43.059164 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a275a-ad83-493e-9b36-405c13b4a54a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:43.059756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a275a-ad83-493e-9b36-405c13b4a54a service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:43.060036 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a275a-ad83-493e-9b36-405c13b4a54a service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:43.060308 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a275a-ad83-493e-9b36-405c13b4a54a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:43.069169 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a275a-ad83-493e-9b36-405c13b4a54a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:43.070653 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a275a-ad83-493e-9b36-405c13b4a54a service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:43.080565 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1364/2752] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:40:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:40:43.251644 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b850de-8c2d-400b-bc3a-867aa97cec08 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:43.253068 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-b3b850de-8c2d-400b-bc3a-867aa97cec08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:43.266571 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b850de-8c2d-400b-bc3a-867aa97cec08 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:43.273058 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8a01e-933b-4fa1-93cf-1a3f2e56d6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:43.273730 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8a01e-933b-4fa1-93cf-1a3f2e56d6ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:43.273977 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8a01e-933b-4fa1-93cf-1a3f2e56d6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:43.274404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8a01e-933b-4fa1-93cf-1a3f2e56d6ac admin admin] PATH_INFO: `/v3/projects/4c206e1031144f509a8957ad347ac345` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:43.275503 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da8a01e-933b-4fa1-93cf-1a3f2e56d6ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c206e1031144f509a8957ad347ac345)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:43.277996 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da8a01e-933b-4fa1-93cf-1a3f2e56d6ac admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:43.303953 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:43.332680 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7da8a01e-933b-4fa1-93cf-1a3f2e56d6ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c206e1031144f509a8957ad347ac345', 'request_id': 'req-7da8a01e-933b-4fa1-93cf-1a3f2e56d6ac'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:43.339540 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7da8a01e-933b-4fa1-93cf-1a3f2e56d6ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c206e1031144f509a8957ad347ac345', 'request_id': 'req-7da8a01e-933b-4fa1-93cf-1a3f2e56d6ac'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:43.347012 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7da8a01e-933b-4fa1-93cf-1a3f2e56d6ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c206e1031144f509a8957ad347ac345', 'request_id': 'req-7da8a01e-933b-4fa1-93cf-1a3f2e56d6ac'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:43.357286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7da8a01e-933b-4fa1-93cf-1a3f2e56d6ac admin admin] The token cache is being invalidate because project 4c206e1031144f509a8957ad347ac345 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:40:43.357931 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7da8a01e-933b-4fa1-93cf-1a3f2e56d6ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:43.358610 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7da8a01e-933b-4fa1-93cf-1a3f2e56d6ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:43.360079 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1389/2753] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:40:43 2022] DELETE /identity/v3/projects/4c206e1031144f509a8957ad347ac345 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:48.082251 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a275a-ad83-493e-9b36-405c13b4a54a service neutron] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:48.083739 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-eb4a275a-ad83-493e-9b36-405c13b4a54a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:48.096550 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a275a-ad83-493e-9b36-405c13b4a54a service neutron] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:48.130464 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:48.154561 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f4b7b-9909-4527-9b77-20143f55581c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:48.155190 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f4b7b-9909-4527-9b77-20143f55581c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:48.155464 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f4b7b-9909-4527-9b77-20143f55581c admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:48.155735 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f4b7b-9909-4527-9b77-20143f55581c admin admin] PATH_INFO: `/v3/users/d951450c1004415bade76d83e7d220e2` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:48.156855 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f4b7b-9909-4527-9b77-20143f55581c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d951450c1004415bade76d83e7d220e2)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:48.158791 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f4b7b-9909-4527-9b77-20143f55581c admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:48.205429 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-341f4b7b-9909-4527-9b77-20143f55581c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd951450c1004415bade76d83e7d220e2'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:48.206102 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-341f4b7b-9909-4527-9b77-20143f55581c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd951450c1004415bade76d83e7d220e2'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:48.206710 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-341f4b7b-9909-4527-9b77-20143f55581c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd951450c1004415bade76d83e7d220e2'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:48.213580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-341f4b7b-9909-4527-9b77-20143f55581c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd951450c1004415bade76d83e7d220e2'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:48.225187 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-341f4b7b-9909-4527-9b77-20143f55581c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd951450c1004415bade76d83e7d220e2'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:48.238282 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1365/2754] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:40:48 2022] DELETE /identity/v3/users/d951450c1004415bade76d83e7d220e2 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:48.242771 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084e8ac-57f1-44bc-b491-f5605a87307b None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:48.243292 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084e8ac-57f1-44bc-b491-f5605a87307b None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:48.243292 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084e8ac-57f1-44bc-b491-f5605a87307b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:48.263045 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-c084e8ac-57f1-44bc-b491-f5605a87307b None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:48.264326 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-c084e8ac-57f1-44bc-b491-f5605a87307b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:48.324056 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1390/2755] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:48.333597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f4b7b-9909-4527-9b77-20143f55581c admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:48.335102 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-341f4b7b-9909-4527-9b77-20143f55581c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:48.393601 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f4b7b-9909-4527-9b77-20143f55581c admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:48.397262 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f76b22-3820-4d64-b715-80b0d6eafaa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:48.397874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f76b22-3820-4d64-b715-80b0d6eafaa6 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:48.398187 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f76b22-3820-4d64-b715-80b0d6eafaa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:48.398464 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f76b22-3820-4d64-b715-80b0d6eafaa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:48.407675 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f76b22-3820-4d64-b715-80b0d6eafaa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:48.409089 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f76b22-3820-4d64-b715-80b0d6eafaa6 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:48.419151 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1366/2756] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:40:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:40:48.697253 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084e8ac-57f1-44bc-b491-f5605a87307b None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:48.699272 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-c084e8ac-57f1-44bc-b491-f5605a87307b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:48.712765 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084e8ac-57f1-44bc-b491-f5605a87307b None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:48.718940 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ec317-bb02-4204-9194-47482e6623dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:48.719590 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ec317-bb02-4204-9194-47482e6623dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:48.719872 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ec317-bb02-4204-9194-47482e6623dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:48.720148 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ec317-bb02-4204-9194-47482e6623dc admin admin] PATH_INFO: `/v3/projects/a798ba18b97d4cd7a5ed24223870c0e7` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:48.721317 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ec317-bb02-4204-9194-47482e6623dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a798ba18b97d4cd7a5ed24223870c0e7)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:48.723314 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ec317-bb02-4204-9194-47482e6623dc admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:48.779977 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d47ec317-bb02-4204-9194-47482e6623dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a798ba18b97d4cd7a5ed24223870c0e7', 'request_id': 'req-d47ec317-bb02-4204-9194-47482e6623dc'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:48.788127 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d47ec317-bb02-4204-9194-47482e6623dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a798ba18b97d4cd7a5ed24223870c0e7', 'request_id': 'req-d47ec317-bb02-4204-9194-47482e6623dc'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:48.802916 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d47ec317-bb02-4204-9194-47482e6623dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a798ba18b97d4cd7a5ed24223870c0e7', 'request_id': 'req-d47ec317-bb02-4204-9194-47482e6623dc'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:48.819928 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d47ec317-bb02-4204-9194-47482e6623dc admin admin] The token cache is being invalidate because project a798ba18b97d4cd7a5ed24223870c0e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:40:48.820502 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d47ec317-bb02-4204-9194-47482e6623dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:48.821225 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-d47ec317-bb02-4204-9194-47482e6623dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:48.822875 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1391/2757] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:40:48 2022] DELETE /identity/v3/projects/a798ba18b97d4cd7a5ed24223870c0e7 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:48.826875 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f76b22-3820-4d64-b715-80b0d6eafaa6 service neutron] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:48.836544 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f76b22-3820-4d64-b715-80b0d6eafaa6 service neutron] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:48.906815 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df58d5-99c2-4947-a7cb-ba48c86ae563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:48.907680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df58d5-99c2-4947-a7cb-ba48c86ae563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:48.908033 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df58d5-99c2-4947-a7cb-ba48c86ae563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:48.908356 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df58d5-99c2-4947-a7cb-ba48c86ae563 admin admin] PATH_INFO: `/v3/users/8a9217595eeb4931ad35b4596a2281b9` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:48.909764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df58d5-99c2-4947-a7cb-ba48c86ae563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a9217595eeb4931ad35b4596a2281b9)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:48.911702 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df58d5-99c2-4947-a7cb-ba48c86ae563 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:48.913708 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:48.966327 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-85df58d5-99c2-4947-a7cb-ba48c86ae563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a9217595eeb4931ad35b4596a2281b9'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:48.966991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-85df58d5-99c2-4947-a7cb-ba48c86ae563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a9217595eeb4931ad35b4596a2281b9'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:48.967683 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-85df58d5-99c2-4947-a7cb-ba48c86ae563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a9217595eeb4931ad35b4596a2281b9'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:48.976083 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-85df58d5-99c2-4947-a7cb-ba48c86ae563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a9217595eeb4931ad35b4596a2281b9'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:48.991380 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-85df58d5-99c2-4947-a7cb-ba48c86ae563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a9217595eeb4931ad35b4596a2281b9'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:49.005269 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1367/2758] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:40:48 2022] DELETE /identity/v3/users/8a9217595eeb4931ad35b4596a2281b9 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:49.272834 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ec317-bb02-4204-9194-47482e6623dc admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:49.274352 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-d47ec317-bb02-4204-9194-47482e6623dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:49.292059 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ec317-bb02-4204-9194-47482e6623dc admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:49.326774 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:49.353591 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e1b7f-9911-4b36-a909-d166401e9a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:49.354352 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e1b7f-9911-4b36-a909-d166401e9a95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:49.354637 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e1b7f-9911-4b36-a909-d166401e9a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:49.354943 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e1b7f-9911-4b36-a909-d166401e9a95 admin admin] PATH_INFO: `/v3/projects/1c48d105fbcc4bd48caebe7742abe2ae` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:49.356070 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e1b7f-9911-4b36-a909-d166401e9a95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c48d105fbcc4bd48caebe7742abe2ae)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:49.358282 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e1b7f-9911-4b36-a909-d166401e9a95 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:49.404252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-371e1b7f-9911-4b36-a909-d166401e9a95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c48d105fbcc4bd48caebe7742abe2ae', 'request_id': 'req-371e1b7f-9911-4b36-a909-d166401e9a95'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:49.410286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-371e1b7f-9911-4b36-a909-d166401e9a95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c48d105fbcc4bd48caebe7742abe2ae', 'request_id': 'req-371e1b7f-9911-4b36-a909-d166401e9a95'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:49.417757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-371e1b7f-9911-4b36-a909-d166401e9a95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c48d105fbcc4bd48caebe7742abe2ae', 'request_id': 'req-371e1b7f-9911-4b36-a909-d166401e9a95'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:49.428324 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-371e1b7f-9911-4b36-a909-d166401e9a95 admin admin] The token cache is being invalidate because project 1c48d105fbcc4bd48caebe7742abe2ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:40:49.428610 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-371e1b7f-9911-4b36-a909-d166401e9a95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:49.429293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-371e1b7f-9911-4b36-a909-d166401e9a95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:49.430802 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1392/2759] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:40:49 2022] DELETE /identity/v3/projects/1c48d105fbcc4bd48caebe7742abe2ae => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:49.437292 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93bd7f-5d19-425c-b680-0a4509f2c143 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:49.437773 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93bd7f-5d19-425c-b680-0a4509f2c143 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:49.437889 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93bd7f-5d19-425c-b680-0a4509f2c143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:49.457709 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-7a93bd7f-5d19-425c-b680-0a4509f2c143 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:49.458949 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-7a93bd7f-5d19-425c-b680-0a4509f2c143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:49.517642 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1368/2760] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:49.525960 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d55029-9dbc-43bc-8691-ff575c14e8fa None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:49.526278 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d55029-9dbc-43bc-8691-ff575c14e8fa None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:49.526559 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d55029-9dbc-43bc-8691-ff575c14e8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:49.546390 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-03d55029-9dbc-43bc-8691-ff575c14e8fa None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:49.547597 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-03d55029-9dbc-43bc-8691-ff575c14e8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:49.555479 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1393/2761] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:49.561383 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93bd7f-5d19-425c-b680-0a4509f2c143 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:49.571113 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93bd7f-5d19-425c-b680-0a4509f2c143 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:49.576317 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d50216-a1bd-4c40-85b0-270e01f2e87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:49.576911 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d50216-a1bd-4c40-85b0-270e01f2e87d admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:49.577188 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d50216-a1bd-4c40-85b0-270e01f2e87d admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:49.577458 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d50216-a1bd-4c40-85b0-270e01f2e87d admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:49.578147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d50216-a1bd-4c40-85b0-270e01f2e87d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:40:49.578452 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d50216-a1bd-4c40-85b0-270e01f2e87d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:49.580043 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d50216-a1bd-4c40-85b0-270e01f2e87d admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:49.591522 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1369/2762] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:40:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:40:49.595740 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d55029-9dbc-43bc-8691-ff575c14e8fa None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:49.605094 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d55029-9dbc-43bc-8691-ff575c14e8fa None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:49.610062 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26285b83-b5b1-4273-ac46-0c3e56c4898e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:49.610713 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26285b83-b5b1-4273-ac46-0c3e56c4898e admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:49.610993 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26285b83-b5b1-4273-ac46-0c3e56c4898e admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:49.611269 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26285b83-b5b1-4273-ac46-0c3e56c4898e admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:49.611914 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26285b83-b5b1-4273-ac46-0c3e56c4898e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:49.613838 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26285b83-b5b1-4273-ac46-0c3e56c4898e admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:49.643929 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1394/2763] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:40:49 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:40:49.647452 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d50216-a1bd-4c40-85b0-270e01f2e87d admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:49.657417 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d50216-a1bd-4c40-85b0-270e01f2e87d admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:49.715187 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be34143-c47d-4e39-bb3b-6fbaeafd3e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:49.715765 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be34143-c47d-4e39-bb3b-6fbaeafd3e8d admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:49.716047 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be34143-c47d-4e39-bb3b-6fbaeafd3e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:49.716317 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be34143-c47d-4e39-bb3b-6fbaeafd3e8d admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:49.716948 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be34143-c47d-4e39-bb3b-6fbaeafd3e8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:49.718901 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be34143-c47d-4e39-bb3b-6fbaeafd3e8d admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:49.730359 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:49.743137 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1370/2764] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:40:49 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:40:49.747592 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26285b83-b5b1-4273-ac46-0c3e56c4898e admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:49.757890 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26285b83-b5b1-4273-ac46-0c3e56c4898e admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:49.762546 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3434ea7-5e24-40c6-9559-d537fc80f203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:49.763167 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3434ea7-5e24-40c6-9559-d537fc80f203 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:49.763448 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3434ea7-5e24-40c6-9559-d537fc80f203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:49.763723 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3434ea7-5e24-40c6-9559-d537fc80f203 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:49.764359 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3434ea7-5e24-40c6-9559-d537fc80f203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:49.766106 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3434ea7-5e24-40c6-9559-d537fc80f203 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:49.776098 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1395/2765] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:40:49.780005 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be34143-c47d-4e39-bb3b-6fbaeafd3e8d admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:49.789865 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be34143-c47d-4e39-bb3b-6fbaeafd3e8d admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:49.795200 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15367b6b-1da8-440c-be06-0d5c07e570d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:49.795809 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15367b6b-1da8-440c-be06-0d5c07e570d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:49.796090 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15367b6b-1da8-440c-be06-0d5c07e570d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:49.796392 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15367b6b-1da8-440c-be06-0d5c07e570d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:49.797000 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15367b6b-1da8-440c-be06-0d5c07e570d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:49.798632 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15367b6b-1da8-440c-be06-0d5c07e570d6 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:49.810666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1371/2766] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:40:49.814783 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3434ea7-5e24-40c6-9559-d537fc80f203 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:49.824659 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3434ea7-5e24-40c6-9559-d537fc80f203 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:49.829237 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573fecb-bb33-4654-8c91-9cc9f22ffec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:49.829870 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573fecb-bb33-4654-8c91-9cc9f22ffec6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:49.830192 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573fecb-bb33-4654-8c91-9cc9f22ffec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:49.830465 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573fecb-bb33-4654-8c91-9cc9f22ffec6 admin admin] PATH_INFO: `/v3/projects/cd962098de4f4875836059412d4faf80/users/ae954c130c834fad9ab6f817b3866444/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:49.845085 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6573fecb-bb33-4654-8c91-9cc9f22ffec6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd962098de4f4875836059412d4faf80, user_id=ae954c130c834fad9ab6f817b3866444, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:49.847781 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6573fecb-bb33-4654-8c91-9cc9f22ffec6 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:49.861156 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1396/2767] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:40:49 2022] PUT /identity/v3/projects/cd962098de4f4875836059412d4faf80/users/ae954c130c834fad9ab6f817b3866444/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:49.872917 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc4e77-ed06-4f17-bbc7-9770cf716973 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:49.873203 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc4e77-ed06-4f17-bbc7-9770cf716973 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:49.873472 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc4e77-ed06-4f17-bbc7-9770cf716973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:49.900016 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-b4cc4e77-ed06-4f17-bbc7-9770cf716973 None None] MFA Rules not processed for user `ae954c130c834fad9ab6f817b3866444`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:49.901228 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-b4cc4e77-ed06-4f17-bbc7-9770cf716973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:49.965115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1372/2768] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:40:49.976046 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573fecb-bb33-4654-8c91-9cc9f22ffec6 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:49.977708 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-6573fecb-bb33-4654-8c91-9cc9f22ffec6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:50.035521 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573fecb-bb33-4654-8c91-9cc9f22ffec6 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:50.039123 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4e2009-b975-4a0b-bec8-fc2ab50ffbd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:50.039719 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e2009-b975-4a0b-bec8-fc2ab50ffbd8 service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:50.039992 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e2009-b975-4a0b-bec8-fc2ab50ffbd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:50.040260 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e2009-b975-4a0b-bec8-fc2ab50ffbd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:50.048888 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4e2009-b975-4a0b-bec8-fc2ab50ffbd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:50.050318 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4e2009-b975-4a0b-bec8-fc2ab50ffbd8 service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:50.059601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1397/2769] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:40:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:40:51.812615 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc4e77-ed06-4f17-bbc7-9770cf716973 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:51.814121 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-b4cc4e77-ed06-4f17-bbc7-9770cf716973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:51.903201 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc4e77-ed06-4f17-bbc7-9770cf716973 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:51.908580 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe31778-9242-455d-b81a-4ba7f6270488 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84b02e7351914cabb67462454b328d05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:51.909174 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe31778-9242-455d-b81a-4ba7f6270488 service placement] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:51.909449 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe31778-9242-455d-b81a-4ba7f6270488 service placement] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:51.909717 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe31778-9242-455d-b81a-4ba7f6270488 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:51.911448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-3fe31778-9242-455d-b81a-4ba7f6270488 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:51.922642 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe31778-9242-455d-b81a-4ba7f6270488 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:51.924183 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe31778-9242-455d-b81a-4ba7f6270488 service placement] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:51.934615 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1373/2770] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:40:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:40:58.484791 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4e2009-b975-4a0b-bec8-fc2ab50ffbd8 service cinder] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:58.490074 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:58.548294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4e2009-b975-4a0b-bec8-fc2ab50ffbd8 service cinder] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:58.552794 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1169e379-cec9-444f-8023-305a253a9455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:58.553407 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1169e379-cec9-444f-8023-305a253a9455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:58.553688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1169e379-cec9-444f-8023-305a253a9455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:58.553964 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1169e379-cec9-444f-8023-305a253a9455 admin admin] PATH_INFO: `/v3/users/ae954c130c834fad9ab6f817b3866444` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:58.555105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1169e379-cec9-444f-8023-305a253a9455 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae954c130c834fad9ab6f817b3866444)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:58.557004 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1169e379-cec9-444f-8023-305a253a9455 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:58.603314 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-1169e379-cec9-444f-8023-305a253a9455 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae954c130c834fad9ab6f817b3866444'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:58.603968 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-1169e379-cec9-444f-8023-305a253a9455 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae954c130c834fad9ab6f817b3866444'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:58.611188 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-1169e379-cec9-444f-8023-305a253a9455 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae954c130c834fad9ab6f817b3866444'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:58.624375 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-1169e379-cec9-444f-8023-305a253a9455 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae954c130c834fad9ab6f817b3866444'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:58.625035 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-1169e379-cec9-444f-8023-305a253a9455 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae954c130c834fad9ab6f817b3866444'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:58.638238 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1398/2771] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:40:58 2022] DELETE /identity/v3/users/ae954c130c834fad9ab6f817b3866444 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:58.642692 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce79da0-c6ef-43de-8b50-0ebbe1a79703 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:58.643155 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce79da0-c6ef-43de-8b50-0ebbe1a79703 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:58.643279 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce79da0-c6ef-43de-8b50-0ebbe1a79703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:58.663369 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-7ce79da0-c6ef-43de-8b50-0ebbe1a79703 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:58.664577 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-7ce79da0-c6ef-43de-8b50-0ebbe1a79703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:58.706485 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=100 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:58.727356 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1374/2772] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:58.737923 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1169e379-cec9-444f-8023-305a253a9455 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:58.739393 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-1169e379-cec9-444f-8023-305a253a9455 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:58.826198 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1169e379-cec9-444f-8023-305a253a9455 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:58.830924 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a981511-7f4a-46e0-af2b-b63239653b3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:58.831533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a981511-7f4a-46e0-af2b-b63239653b3c service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:58.831813 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a981511-7f4a-46e0-af2b-b63239653b3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:58.832249 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a981511-7f4a-46e0-af2b-b63239653b3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:58.842675 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a981511-7f4a-46e0-af2b-b63239653b3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:58.844113 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a981511-7f4a-46e0-af2b-b63239653b3c service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:58.854424 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1399/2773] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:40:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:40:59.119592 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce79da0-c6ef-43de-8b50-0ebbe1a79703 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:59.121074 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-7ce79da0-c6ef-43de-8b50-0ebbe1a79703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:59.133351 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce79da0-c6ef-43de-8b50-0ebbe1a79703 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:59.138214 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34abc52a-0234-4a04-ac71-3aecd0bcc987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:59.138918 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34abc52a-0234-4a04-ac71-3aecd0bcc987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:59.139212 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34abc52a-0234-4a04-ac71-3aecd0bcc987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:59.139476 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34abc52a-0234-4a04-ac71-3aecd0bcc987 admin admin] PATH_INFO: `/v3/projects/cd962098de4f4875836059412d4faf80` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:59.140647 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34abc52a-0234-4a04-ac71-3aecd0bcc987 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd962098de4f4875836059412d4faf80)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:59.142526 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34abc52a-0234-4a04-ac71-3aecd0bcc987 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:59.189502 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-34abc52a-0234-4a04-ac71-3aecd0bcc987 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd962098de4f4875836059412d4faf80', 'request_id': 'req-34abc52a-0234-4a04-ac71-3aecd0bcc987'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:59.195670 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-34abc52a-0234-4a04-ac71-3aecd0bcc987 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd962098de4f4875836059412d4faf80', 'request_id': 'req-34abc52a-0234-4a04-ac71-3aecd0bcc987'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:59.201519 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-34abc52a-0234-4a04-ac71-3aecd0bcc987 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd962098de4f4875836059412d4faf80', 'request_id': 'req-34abc52a-0234-4a04-ac71-3aecd0bcc987'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:59.213121 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-34abc52a-0234-4a04-ac71-3aecd0bcc987 admin admin] The token cache is being invalidate because project cd962098de4f4875836059412d4faf80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:40:59.213404 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-34abc52a-0234-4a04-ac71-3aecd0bcc987 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:59.214061 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-34abc52a-0234-4a04-ac71-3aecd0bcc987 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:40:59.216048 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1375/2774] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:40:59 2022] DELETE /identity/v3/projects/cd962098de4f4875836059412d4faf80 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:59.220897 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cae1ad-861e-4e54-b0cd-adb27013a481 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:59.221188 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cae1ad-861e-4e54-b0cd-adb27013a481 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:59.221466 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cae1ad-861e-4e54-b0cd-adb27013a481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:59.236125 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:59.242258 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-67cae1ad-861e-4e54-b0cd-adb27013a481 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:59.243901 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-67cae1ad-861e-4e54-b0cd-adb27013a481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:59.303300 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1400/2775] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:59.312107 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cbd8a-39f4-4b65-8b3c-05c61f296904 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:59.312385 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cbd8a-39f4-4b65-8b3c-05c61f296904 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:59.312734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cbd8a-39f4-4b65-8b3c-05c61f296904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:59.331979 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-7f3cbd8a-39f4-4b65-8b3c-05c61f296904 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:59.333171 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-7f3cbd8a-39f4-4b65-8b3c-05c61f296904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:59.342134 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1376/2776] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:59.346313 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cae1ad-861e-4e54-b0cd-adb27013a481 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:59.356710 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cae1ad-861e-4e54-b0cd-adb27013a481 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:59.361559 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1213db8-046d-4d81-905d-ffe19fc7af09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:59.362197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1213db8-046d-4d81-905d-ffe19fc7af09 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:59.362481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1213db8-046d-4d81-905d-ffe19fc7af09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:59.362755 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1213db8-046d-4d81-905d-ffe19fc7af09 admin admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:59.363400 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1213db8-046d-4d81-905d-ffe19fc7af09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:40:59.363701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1213db8-046d-4d81-905d-ffe19fc7af09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:59.365291 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1213db8-046d-4d81-905d-ffe19fc7af09 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:59.376927 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1401/2777] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:40:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:40:59.381039 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cbd8a-39f4-4b65-8b3c-05c61f296904 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:59.390328 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cbd8a-39f4-4b65-8b3c-05c61f296904 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:59.394888 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdcce88-597e-40fe-99e3-681bf126d1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:59.395475 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcce88-597e-40fe-99e3-681bf126d1d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:59.395752 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcce88-597e-40fe-99e3-681bf126d1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:59.396021 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcce88-597e-40fe-99e3-681bf126d1d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:59.396679 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdcce88-597e-40fe-99e3-681bf126d1d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:59.398592 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdcce88-597e-40fe-99e3-681bf126d1d2 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:59.426160 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1377/2778] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:40:59 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:40:59.430348 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1213db8-046d-4d81-905d-ffe19fc7af09 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:59.440695 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1213db8-046d-4d81-905d-ffe19fc7af09 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:59.497503 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181a169-f5d3-4097-8b18-9e988784f708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:59.498144 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181a169-f5d3-4097-8b18-9e988784f708 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:59.498353 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181a169-f5d3-4097-8b18-9e988784f708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:59.498626 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181a169-f5d3-4097-8b18-9e988784f708 admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:59.499262 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6181a169-f5d3-4097-8b18-9e988784f708 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:59.501166 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6181a169-f5d3-4097-8b18-9e988784f708 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:59.522795 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1402/2779] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:40:59 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:40:59.527211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdcce88-597e-40fe-99e3-681bf126d1d2 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:59.537376 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdcce88-597e-40fe-99e3-681bf126d1d2 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:59.541983 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92564ce-b69a-4f24-b263-a7ccff74e008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:59.543099 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92564ce-b69a-4f24-b263-a7ccff74e008 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:59.543371 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92564ce-b69a-4f24-b263-a7ccff74e008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:59.543639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92564ce-b69a-4f24-b263-a7ccff74e008 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:59.544267 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92564ce-b69a-4f24-b263-a7ccff74e008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:59.545831 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92564ce-b69a-4f24-b263-a7ccff74e008 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:59.555493 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1378/2780] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:40:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:40:59.560081 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181a169-f5d3-4097-8b18-9e988784f708 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:59.570095 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181a169-f5d3-4097-8b18-9e988784f708 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:59.574701 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22303a-fd95-4d54-bb05-e6c903e21cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:59.575289 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22303a-fd95-4d54-bb05-e6c903e21cb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:59.575568 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22303a-fd95-4d54-bb05-e6c903e21cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:59.575841 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22303a-fd95-4d54-bb05-e6c903e21cb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:59.576470 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f22303a-fd95-4d54-bb05-e6c903e21cb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:59.578078 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f22303a-fd95-4d54-bb05-e6c903e21cb1 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:59.588649 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1403/2781] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:40:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:40:59.592667 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92564ce-b69a-4f24-b263-a7ccff74e008 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:59.602820 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92564ce-b69a-4f24-b263-a7ccff74e008 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:59.608139 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32fc75-7ce4-469f-ba05-ce68aaece95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:59.608771 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32fc75-7ce4-469f-ba05-ce68aaece95f admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:59.609049 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32fc75-7ce4-469f-ba05-ce68aaece95f admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:59.609324 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32fc75-7ce4-469f-ba05-ce68aaece95f admin admin] PATH_INFO: `/v3/projects/af5f82a2a17043078467ed7c8d2f0bf0/users/95663e6d743745d5b609cbc46f73ce06/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:59.625214 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32fc75-7ce4-469f-ba05-ce68aaece95f admin admin] RBAC: Authorizing `identity:create_grant(project_id=af5f82a2a17043078467ed7c8d2f0bf0, user_id=95663e6d743745d5b609cbc46f73ce06, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:59.627913 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32fc75-7ce4-469f-ba05-ce68aaece95f admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:59.641055 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1379/2782] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:40:59 2022] PUT /identity/v3/projects/af5f82a2a17043078467ed7c8d2f0bf0/users/95663e6d743745d5b609cbc46f73ce06/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:40:59.652181 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501301d-a5fe-448f-ac47-c2eefb9961ad None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:59.652473 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501301d-a5fe-448f-ac47-c2eefb9961ad None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:59.652733 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501301d-a5fe-448f-ac47-c2eefb9961ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:59.680517 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-1501301d-a5fe-448f-ac47-c2eefb9961ad None None] MFA Rules not processed for user `95663e6d743745d5b609cbc46f73ce06`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:40:59.681734 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-1501301d-a5fe-448f-ac47-c2eefb9961ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:40:59.745304 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1404/2783] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:40:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:40:59.750450 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32fc75-7ce4-469f-ba05-ce68aaece95f admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:59.812178 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32fc75-7ce4-469f-ba05-ce68aaece95f admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:59.816845 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4d867-5bec-4719-a814-b542a488f7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:59.817445 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4d867-5bec-4719-a814-b542a488f7dc admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:59.817727 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4d867-5bec-4719-a814-b542a488f7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:59.818000 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4d867-5bec-4719-a814-b542a488f7dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:59.818667 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b4d867-5bec-4719-a814-b542a488f7dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:59.821392 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b4d867-5bec-4719-a814-b542a488f7dc admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:59.830780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:59.849514 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1380/2784] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:40:59 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:40:59.853533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501301d-a5fe-448f-ac47-c2eefb9961ad None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:59.859859 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:40:59.866945 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501301d-a5fe-448f-ac47-c2eefb9961ad None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:59.923943 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d49a0-62aa-4dd9-991c-5bcf28281291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:59.924558 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d49a0-62aa-4dd9-991c-5bcf28281291 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:59.924837 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d49a0-62aa-4dd9-991c-5bcf28281291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:59.925108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d49a0-62aa-4dd9-991c-5bcf28281291 admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:59.925743 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d49a0-62aa-4dd9-991c-5bcf28281291 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:59.927690 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d49a0-62aa-4dd9-991c-5bcf28281291 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:59.948179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1405/2785] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:40:59 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:40:59.952306 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4d867-5bec-4719-a814-b542a488f7dc admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:59.962605 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4d867-5bec-4719-a814-b542a488f7dc admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:40:59.967801 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd70411-29f3-4007-9f66-c25825b2feb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:40:59.968393 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd70411-29f3-4007-9f66-c25825b2feb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:40:59.968671 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd70411-29f3-4007-9f66-c25825b2feb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:40:59.968946 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd70411-29f3-4007-9f66-c25825b2feb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:40:59.969607 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd70411-29f3-4007-9f66-c25825b2feb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:40:59.971247 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd70411-29f3-4007-9f66-c25825b2feb1 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:40:59.981021 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1381/2786] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:40:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:40:59.985143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d49a0-62aa-4dd9-991c-5bcf28281291 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:40:59.996096 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d49a0-62aa-4dd9-991c-5bcf28281291 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:00.000749 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f3faf-08f8-4782-a324-2f013be655ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:00.001337 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f3faf-08f8-4782-a324-2f013be655ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:00.001616 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f3faf-08f8-4782-a324-2f013be655ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:00.001889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f3faf-08f8-4782-a324-2f013be655ef admin admin] PATH_INFO: `/v3/projects/ef77986f7ba6471b9042fdc90e9cc6a0/users/a49fdfffa03941d4a2cac7cce9ec220d/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:00.019256 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f3faf-08f8-4782-a324-2f013be655ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef77986f7ba6471b9042fdc90e9cc6a0, user_id=a49fdfffa03941d4a2cac7cce9ec220d, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:00.021991 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f3faf-08f8-4782-a324-2f013be655ef admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:00.035544 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1406/2787] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:40:59 2022] PUT /identity/v3/projects/ef77986f7ba6471b9042fdc90e9cc6a0/users/a49fdfffa03941d4a2cac7cce9ec220d/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:41:00.039451 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd70411-29f3-4007-9f66-c25825b2feb1 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:00.049755 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd70411-29f3-4007-9f66-c25825b2feb1 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:00.108390 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa52b92-2bc3-4129-b890-8f17c160dc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:00.109001 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa52b92-2bc3-4129-b890-8f17c160dc70 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:00.109279 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa52b92-2bc3-4129-b890-8f17c160dc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:00.109609 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa52b92-2bc3-4129-b890-8f17c160dc70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:00.110276 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa52b92-2bc3-4129-b890-8f17c160dc70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:00.111859 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa52b92-2bc3-4129-b890-8f17c160dc70 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:00.122371 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1382/2788] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:41:00 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:41:00.126576 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f3faf-08f8-4782-a324-2f013be655ef admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:00.137547 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f3faf-08f8-4782-a324-2f013be655ef admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:00.142278 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2659d1f3-6fbc-4027-9b8c-f65f57b10ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:00.143003 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659d1f3-6fbc-4027-9b8c-f65f57b10ba0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:00.143352 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659d1f3-6fbc-4027-9b8c-f65f57b10ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:00.143723 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659d1f3-6fbc-4027-9b8c-f65f57b10ba0 admin admin] PATH_INFO: `/v3/projects/ef77986f7ba6471b9042fdc90e9cc6a0/users/a49fdfffa03941d4a2cac7cce9ec220d/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:00.145733 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2659d1f3-6fbc-4027-9b8c-f65f57b10ba0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef77986f7ba6471b9042fdc90e9cc6a0, user_id=a49fdfffa03941d4a2cac7cce9ec220d, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:00.148498 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2659d1f3-6fbc-4027-9b8c-f65f57b10ba0 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:00.159416 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1407/2789] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:41:00 2022] PUT /identity/v3/projects/ef77986f7ba6471b9042fdc90e9cc6a0/users/a49fdfffa03941d4a2cac7cce9ec220d/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:41:00.169705 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b88821-2f45-4b07-a486-5bd0210ddf23 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:00.169997 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b88821-2f45-4b07-a486-5bd0210ddf23 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:00.170371 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b88821-2f45-4b07-a486-5bd0210ddf23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:00.201859 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-b2b88821-2f45-4b07-a486-5bd0210ddf23 None None] MFA Rules not processed for user `a49fdfffa03941d4a2cac7cce9ec220d`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:41:00.203100 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-b2b88821-2f45-4b07-a486-5bd0210ddf23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:00.270204 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1383/2790] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:41:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:41:00.281084 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2659d1f3-6fbc-4027-9b8c-f65f57b10ba0 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:00.282860 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-2659d1f3-6fbc-4027-9b8c-f65f57b10ba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:00.340305 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2659d1f3-6fbc-4027-9b8c-f65f57b10ba0 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:00.344028 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98e8fe-97fd-4a54-ba2d-e5f951050171 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:00.344624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98e8fe-97fd-4a54-ba2d-e5f951050171 service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:00.344903 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98e8fe-97fd-4a54-ba2d-e5f951050171 service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:00.345175 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98e8fe-97fd-4a54-ba2d-e5f951050171 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:00.401102 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c98e8fe-97fd-4a54-ba2d-e5f951050171 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:00.402596 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c98e8fe-97fd-4a54-ba2d-e5f951050171 service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:00.411357 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1408/2791] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:41:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:41:02.958253 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b88821-2f45-4b07-a486-5bd0210ddf23 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:03.020138 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b88821-2f45-4b07-a486-5bd0210ddf23 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:03.024613 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21e4b3-22f2-45c3-8d2b-ce0c1d243974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:03.025232 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21e4b3-22f2-45c3-8d2b-ce0c1d243974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:03.025512 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21e4b3-22f2-45c3-8d2b-ce0c1d243974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:03.025789 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21e4b3-22f2-45c3-8d2b-ce0c1d243974 admin admin] PATH_INFO: `/v3/users/95663e6d743745d5b609cbc46f73ce06` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:03.026924 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be21e4b3-22f2-45c3-8d2b-ce0c1d243974 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95663e6d743745d5b609cbc46f73ce06)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:03.028828 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be21e4b3-22f2-45c3-8d2b-ce0c1d243974 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:03.030354 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:41:03.076525 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-be21e4b3-22f2-45c3-8d2b-ce0c1d243974 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95663e6d743745d5b609cbc46f73ce06'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:03.077178 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-be21e4b3-22f2-45c3-8d2b-ce0c1d243974 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95663e6d743745d5b609cbc46f73ce06'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:03.077774 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-be21e4b3-22f2-45c3-8d2b-ce0c1d243974 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95663e6d743745d5b609cbc46f73ce06'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:03.085240 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-be21e4b3-22f2-45c3-8d2b-ce0c1d243974 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95663e6d743745d5b609cbc46f73ce06'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:03.097238 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-be21e4b3-22f2-45c3-8d2b-ce0c1d243974 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95663e6d743745d5b609cbc46f73ce06'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:03.110029 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1384/2792] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:41:02 2022] DELETE /identity/v3/users/95663e6d743745d5b609cbc46f73ce06 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:41:03.114600 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9258bb8-d9c7-47f0-96c5-440c64d5da99 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:03.115128 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9258bb8-d9c7-47f0-96c5-440c64d5da99 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:03.115128 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9258bb8-d9c7-47f0-96c5-440c64d5da99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:03.133937 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-f9258bb8-d9c7-47f0-96c5-440c64d5da99 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:41:03.135143 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-f9258bb8-d9c7-47f0-96c5-440c64d5da99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:03.139632 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:41:03.194450 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1409/2793] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:41:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:41:03.204899 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21e4b3-22f2-45c3-8d2b-ce0c1d243974 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:03.206336 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-be21e4b3-22f2-45c3-8d2b-ce0c1d243974 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:03.262974 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21e4b3-22f2-45c3-8d2b-ce0c1d243974 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:03.266779 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddbaf26-741b-41d1-9fa7-9a4a2af78ca3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:03.267390 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddbaf26-741b-41d1-9fa7-9a4a2af78ca3 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:03.267767 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddbaf26-741b-41d1-9fa7-9a4a2af78ca3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:03.268044 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddbaf26-741b-41d1-9fa7-9a4a2af78ca3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:03.277367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddbaf26-741b-41d1-9fa7-9a4a2af78ca3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:03.278815 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddbaf26-741b-41d1-9fa7-9a4a2af78ca3 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:03.288059 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1385/2794] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:41:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:41:03.554792 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9258bb8-d9c7-47f0-96c5-440c64d5da99 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:03.556311 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-f9258bb8-d9c7-47f0-96c5-440c64d5da99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:03.569088 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9258bb8-d9c7-47f0-96c5-440c64d5da99 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:03.573703 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d211c-2983-49d2-a986-6e81a7f16c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:03.574370 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d211c-2983-49d2-a986-6e81a7f16c1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:03.574681 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d211c-2983-49d2-a986-6e81a7f16c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:03.574917 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d211c-2983-49d2-a986-6e81a7f16c1d admin admin] PATH_INFO: `/v3/projects/af5f82a2a17043078467ed7c8d2f0bf0` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:03.576038 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d211c-2983-49d2-a986-6e81a7f16c1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=af5f82a2a17043078467ed7c8d2f0bf0)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:03.578087 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d211c-2983-49d2-a986-6e81a7f16c1d admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:03.625847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-778d211c-2983-49d2-a986-6e81a7f16c1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af5f82a2a17043078467ed7c8d2f0bf0', 'request_id': 'req-778d211c-2983-49d2-a986-6e81a7f16c1d'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:03.632674 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-778d211c-2983-49d2-a986-6e81a7f16c1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af5f82a2a17043078467ed7c8d2f0bf0', 'request_id': 'req-778d211c-2983-49d2-a986-6e81a7f16c1d'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:03.641860 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-778d211c-2983-49d2-a986-6e81a7f16c1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af5f82a2a17043078467ed7c8d2f0bf0', 'request_id': 'req-778d211c-2983-49d2-a986-6e81a7f16c1d'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:03.653383 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-778d211c-2983-49d2-a986-6e81a7f16c1d admin admin] The token cache is being invalidate because project af5f82a2a17043078467ed7c8d2f0bf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:41:03.653670 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-778d211c-2983-49d2-a986-6e81a7f16c1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:03.654329 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-778d211c-2983-49d2-a986-6e81a7f16c1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:03.655816 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1410/2795] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:41:03 2022] DELETE /identity/v3/projects/af5f82a2a17043078467ed7c8d2f0bf0 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:41:03.660466 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddbaf26-741b-41d1-9fa7-9a4a2af78ca3 service neutron] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:03.673726 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddbaf26-741b-41d1-9fa7-9a4a2af78ca3 service neutron] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:03.727283 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6939e3-4564-452a-ab75-f99e9cd85e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:03.727907 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6939e3-4564-452a-ab75-f99e9cd85e00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:03.728183 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6939e3-4564-452a-ab75-f99e9cd85e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:03.728454 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6939e3-4564-452a-ab75-f99e9cd85e00 admin admin] PATH_INFO: `/v3/users/a49fdfffa03941d4a2cac7cce9ec220d` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:03.729623 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6939e3-4564-452a-ab75-f99e9cd85e00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a49fdfffa03941d4a2cac7cce9ec220d)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:03.732065 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6939e3-4564-452a-ab75-f99e9cd85e00 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:03.761343 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:41:03.789364 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6a6939e3-4564-452a-ab75-f99e9cd85e00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a49fdfffa03941d4a2cac7cce9ec220d'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:03.789660 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6a6939e3-4564-452a-ab75-f99e9cd85e00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a49fdfffa03941d4a2cac7cce9ec220d'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:03.790359 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6a6939e3-4564-452a-ab75-f99e9cd85e00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a49fdfffa03941d4a2cac7cce9ec220d'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:03.798366 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6a6939e3-4564-452a-ab75-f99e9cd85e00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a49fdfffa03941d4a2cac7cce9ec220d'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:03.812897 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-6a6939e3-4564-452a-ab75-f99e9cd85e00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a49fdfffa03941d4a2cac7cce9ec220d'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:03.832480 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1386/2796] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:41:03 2022] DELETE /identity/v3/users/a49fdfffa03941d4a2cac7cce9ec220d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:41:04.099733 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d211c-2983-49d2-a986-6e81a7f16c1d admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:04.101204 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-778d211c-2983-49d2-a986-6e81a7f16c1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:04.115382 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d211c-2983-49d2-a986-6e81a7f16c1d admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:04.173300 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfcee68-755b-49bd-a483-a4c392945853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:04.173930 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfcee68-755b-49bd-a483-a4c392945853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:04.174242 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfcee68-755b-49bd-a483-a4c392945853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:04.174518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfcee68-755b-49bd-a483-a4c392945853 admin admin] PATH_INFO: `/v3/projects/ef77986f7ba6471b9042fdc90e9cc6a0` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:04.175631 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfcee68-755b-49bd-a483-a4c392945853 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef77986f7ba6471b9042fdc90e9cc6a0)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:04.177540 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfcee68-755b-49bd-a483-a4c392945853 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:04.214175 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:41:04.225848 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-edfcee68-755b-49bd-a483-a4c392945853 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef77986f7ba6471b9042fdc90e9cc6a0', 'request_id': 'req-edfcee68-755b-49bd-a483-a4c392945853'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:04.231923 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-edfcee68-755b-49bd-a483-a4c392945853 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef77986f7ba6471b9042fdc90e9cc6a0', 'request_id': 'req-edfcee68-755b-49bd-a483-a4c392945853'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:04.239405 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-edfcee68-755b-49bd-a483-a4c392945853 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef77986f7ba6471b9042fdc90e9cc6a0', 'request_id': 'req-edfcee68-755b-49bd-a483-a4c392945853'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:04.249793 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-edfcee68-755b-49bd-a483-a4c392945853 admin admin] The token cache is being invalidate because project ef77986f7ba6471b9042fdc90e9cc6a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:41:04.250105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-edfcee68-755b-49bd-a483-a4c392945853 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:04.250726 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-edfcee68-755b-49bd-a483-a4c392945853 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:04.252197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1411/2797] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:41:04 2022] DELETE /identity/v3/projects/ef77986f7ba6471b9042fdc90e9cc6a0 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:41:04.257171 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b3952-0331-49e1-a94a-591e2162c19b None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:04.257622 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b3952-0331-49e1-a94a-591e2162c19b None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:04.257734 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b3952-0331-49e1-a94a-591e2162c19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:04.278142 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-852b3952-0331-49e1-a94a-591e2162c19b None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:41:04.279355 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-852b3952-0331-49e1-a94a-591e2162c19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:04.337801 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1387/2798] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:41:04.346395 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2556d41-69b7-45cf-a8f5-ca66790fa29f None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:04.346679 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2556d41-69b7-45cf-a8f5-ca66790fa29f None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:04.346955 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2556d41-69b7-45cf-a8f5-ca66790fa29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:04.366747 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-c2556d41-69b7-45cf-a8f5-ca66790fa29f None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:41:04.367944 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-c2556d41-69b7-45cf-a8f5-ca66790fa29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:04.376334 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1412/2799] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:41:04.380511 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852b3952-0331-49e1-a94a-591e2162c19b None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:04.390667 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852b3952-0331-49e1-a94a-591e2162c19b None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:04.395152 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd14c6-5347-45d9-8adf-9e78a7beb892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:04.395731 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd14c6-5347-45d9-8adf-9e78a7beb892 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:04.396005 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd14c6-5347-45d9-8adf-9e78a7beb892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:04.396272 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd14c6-5347-45d9-8adf-9e78a7beb892 admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:04.396909 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd14c6-5347-45d9-8adf-9e78a7beb892 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:41:04.397201 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd14c6-5347-45d9-8adf-9e78a7beb892 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:04.398801 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd14c6-5347-45d9-8adf-9e78a7beb892 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:04.410118 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1388/2800] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:41:04.414379 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2556d41-69b7-45cf-a8f5-ca66790fa29f None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:04.423534 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2556d41-69b7-45cf-a8f5-ca66790fa29f None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:04.428191 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2325315-1608-485e-9313-800c22ce149d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:04.428776 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2325315-1608-485e-9313-800c22ce149d admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:04.429055 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2325315-1608-485e-9313-800c22ce149d admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:04.429327 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2325315-1608-485e-9313-800c22ce149d admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:04.429952 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2325315-1608-485e-9313-800c22ce149d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:04.431864 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2325315-1608-485e-9313-800c22ce149d admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:04.459066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1413/2801] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:41:04 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:41:04.463128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd14c6-5347-45d9-8adf-9e78a7beb892 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:04.472849 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd14c6-5347-45d9-8adf-9e78a7beb892 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:04.529369 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f1cdbd-9bce-4a44-809d-48bcf9b51f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:04.529970 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1cdbd-9bce-4a44-809d-48bcf9b51f8f admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:04.530278 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1cdbd-9bce-4a44-809d-48bcf9b51f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:04.530551 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1cdbd-9bce-4a44-809d-48bcf9b51f8f admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:04.531182 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f1cdbd-9bce-4a44-809d-48bcf9b51f8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:04.533089 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f1cdbd-9bce-4a44-809d-48bcf9b51f8f admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:04.554662 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1389/2802] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:41:04 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:41:04.559193 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2325315-1608-485e-9313-800c22ce149d admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:04.569363 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2325315-1608-485e-9313-800c22ce149d admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:04.574604 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c04829f-b1a9-4c8e-97b6-4eaa4c130a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:04.575210 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04829f-b1a9-4c8e-97b6-4eaa4c130a0a admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:04.575494 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04829f-b1a9-4c8e-97b6-4eaa4c130a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:04.575789 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04829f-b1a9-4c8e-97b6-4eaa4c130a0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:04.576424 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c04829f-b1a9-4c8e-97b6-4eaa4c130a0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:04.578015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c04829f-b1a9-4c8e-97b6-4eaa4c130a0a admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:04.587892 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1414/2803] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:41:04.591598 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f1cdbd-9bce-4a44-809d-48bcf9b51f8f admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:04.603069 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f1cdbd-9bce-4a44-809d-48bcf9b51f8f admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:04.608597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67639994-426f-4660-8b56-6f34bc6b2fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:04.609290 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67639994-426f-4660-8b56-6f34bc6b2fe2 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:04.609615 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67639994-426f-4660-8b56-6f34bc6b2fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:04.609912 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67639994-426f-4660-8b56-6f34bc6b2fe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:04.610720 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67639994-426f-4660-8b56-6f34bc6b2fe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:04.612447 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67639994-426f-4660-8b56-6f34bc6b2fe2 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:04.624741 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1390/2804] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:41:04.629062 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c04829f-b1a9-4c8e-97b6-4eaa4c130a0a admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:04.639601 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c04829f-b1a9-4c8e-97b6-4eaa4c130a0a admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:04.644154 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fbe1fb-ccf7-461c-9600-db1969c13a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:04.644790 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbe1fb-ccf7-461c-9600-db1969c13a2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:04.645066 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbe1fb-ccf7-461c-9600-db1969c13a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:04.645335 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbe1fb-ccf7-461c-9600-db1969c13a2a admin admin] PATH_INFO: `/v3/projects/94708783d62e4a68a0e8ce63ae6ce43a/users/308c51a8edc4494b8c300700c09a9751/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:04.659538 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fbe1fb-ccf7-461c-9600-db1969c13a2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=94708783d62e4a68a0e8ce63ae6ce43a, user_id=308c51a8edc4494b8c300700c09a9751, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:04.662825 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fbe1fb-ccf7-461c-9600-db1969c13a2a admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:04.675703 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1415/2805] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:41:04 2022] PUT /identity/v3/projects/94708783d62e4a68a0e8ce63ae6ce43a/users/308c51a8edc4494b8c300700c09a9751/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:41:04.741232 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024bfcf-f90b-438f-b9f9-0140906a7184 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:04.741509 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024bfcf-f90b-438f-b9f9-0140906a7184 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:04.741779 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024bfcf-f90b-438f-b9f9-0140906a7184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:04.755548 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:41:04.771558 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-b024bfcf-f90b-438f-b9f9-0140906a7184 None None] MFA Rules not processed for user `308c51a8edc4494b8c300700c09a9751`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:41:04.772768 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-b024bfcf-f90b-438f-b9f9-0140906a7184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:04.836901 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1391/2806] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:41:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:41:04.847994 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fbe1fb-ccf7-461c-9600-db1969c13a2a admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:04.849655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-80fbe1fb-ccf7-461c-9600-db1969c13a2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:04.929457 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fbe1fb-ccf7-461c-9600-db1969c13a2a admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:04.933116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47989a63-ee61-4e03-a933-67a11a5e8f3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:04.933708 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47989a63-ee61-4e03-a933-67a11a5e8f3b service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:04.933981 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47989a63-ee61-4e03-a933-67a11a5e8f3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:04.934286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47989a63-ee61-4e03-a933-67a11a5e8f3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:04.942341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47989a63-ee61-4e03-a933-67a11a5e8f3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:04.943751 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47989a63-ee61-4e03-a933-67a11a5e8f3b service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:04.953178 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1416/2807] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:41:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:41:09.019265 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b024bfcf-f90b-438f-b9f9-0140906a7184 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:09.081552 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b024bfcf-f90b-438f-b9f9-0140906a7184 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:09.086047 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0eeda-4082-4670-be3f-569fc1f4358d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:09.086668 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0eeda-4082-4670-be3f-569fc1f4358d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:09.086949 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0eeda-4082-4670-be3f-569fc1f4358d admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:09.087224 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0eeda-4082-4670-be3f-569fc1f4358d admin admin] PATH_INFO: `/v3/users/308c51a8edc4494b8c300700c09a9751` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:09.088339 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb0eeda-4082-4670-be3f-569fc1f4358d admin admin] RBAC: Authorizing `identity:delete_user(user_id=308c51a8edc4494b8c300700c09a9751)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:09.090289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb0eeda-4082-4670-be3f-569fc1f4358d admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:09.136600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-bcb0eeda-4082-4670-be3f-569fc1f4358d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '308c51a8edc4494b8c300700c09a9751'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:09.137252 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-bcb0eeda-4082-4670-be3f-569fc1f4358d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '308c51a8edc4494b8c300700c09a9751'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:09.137865 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-bcb0eeda-4082-4670-be3f-569fc1f4358d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '308c51a8edc4494b8c300700c09a9751'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:09.144981 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-bcb0eeda-4082-4670-be3f-569fc1f4358d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '308c51a8edc4494b8c300700c09a9751'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:09.156720 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-bcb0eeda-4082-4670-be3f-569fc1f4358d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '308c51a8edc4494b8c300700c09a9751'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:09.169306 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1392/2808] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:41:09 2022] DELETE /identity/v3/users/308c51a8edc4494b8c300700c09a9751 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:41:09.173754 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9aa8d8-c123-4710-88ff-4d56b7907218 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:09.174268 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9aa8d8-c123-4710-88ff-4d56b7907218 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:09.174396 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9aa8d8-c123-4710-88ff-4d56b7907218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:09.193270 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-bf9aa8d8-c123-4710-88ff-4d56b7907218 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:41:09.194490 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-bf9aa8d8-c123-4710-88ff-4d56b7907218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:09.233341 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:41:09.255834 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1417/2809] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:41:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:41:09.265428 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0eeda-4082-4670-be3f-569fc1f4358d admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:09.266983 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-bcb0eeda-4082-4670-be3f-569fc1f4358d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:09.285261 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:41:09.326412 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0eeda-4082-4670-be3f-569fc1f4358d admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:09.329969 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b557a70-8278-499a-b782-13aac27efd34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:09.330588 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b557a70-8278-499a-b782-13aac27efd34 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:09.330870 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b557a70-8278-499a-b782-13aac27efd34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:09.331141 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b557a70-8278-499a-b782-13aac27efd34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:09.339936 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b557a70-8278-499a-b782-13aac27efd34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:09.341287 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b557a70-8278-499a-b782-13aac27efd34 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:09.350579 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1393/2810] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:41:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:41:09.614447 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9aa8d8-c123-4710-88ff-4d56b7907218 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:09.616181 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-bf9aa8d8-c123-4710-88ff-4d56b7907218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:09.630758 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9aa8d8-c123-4710-88ff-4d56b7907218 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:09.636919 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a11adb6-3d64-4299-871c-48e807dd7574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:09.637599 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11adb6-3d64-4299-871c-48e807dd7574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:09.637890 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11adb6-3d64-4299-871c-48e807dd7574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:09.638193 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11adb6-3d64-4299-871c-48e807dd7574 admin admin] PATH_INFO: `/v3/projects/94708783d62e4a68a0e8ce63ae6ce43a` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:09.639325 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a11adb6-3d64-4299-871c-48e807dd7574 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94708783d62e4a68a0e8ce63ae6ce43a)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:09.641243 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a11adb6-3d64-4299-871c-48e807dd7574 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:09.693677 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7a11adb6-3d64-4299-871c-48e807dd7574 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94708783d62e4a68a0e8ce63ae6ce43a', 'request_id': 'req-7a11adb6-3d64-4299-871c-48e807dd7574'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:09.700258 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7a11adb6-3d64-4299-871c-48e807dd7574 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94708783d62e4a68a0e8ce63ae6ce43a', 'request_id': 'req-7a11adb6-3d64-4299-871c-48e807dd7574'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:09.707698 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7a11adb6-3d64-4299-871c-48e807dd7574 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94708783d62e4a68a0e8ce63ae6ce43a', 'request_id': 'req-7a11adb6-3d64-4299-871c-48e807dd7574'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:09.719147 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7a11adb6-3d64-4299-871c-48e807dd7574 admin admin] The token cache is being invalidate because project 94708783d62e4a68a0e8ce63ae6ce43a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:41:09.719437 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7a11adb6-3d64-4299-871c-48e807dd7574 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:09.720069 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7a11adb6-3d64-4299-871c-48e807dd7574 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:09.721559 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1418/2811] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:41:09 2022] DELETE /identity/v3/projects/94708783d62e4a68a0e8ce63ae6ce43a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:41:09.726436 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dab2e2-eb12-4d46-852e-26e3d2aec6b4 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:09.726764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dab2e2-eb12-4d46-852e-26e3d2aec6b4 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:09.726996 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dab2e2-eb12-4d46-852e-26e3d2aec6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:09.747276 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-59dab2e2-eb12-4d46-852e-26e3d2aec6b4 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:41:09.748484 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-59dab2e2-eb12-4d46-852e-26e3d2aec6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:09.812170 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1394/2812] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:41:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:41:09.819924 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf59f9-ccf2-43f1-9a40-13e8f073ee78 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:09.820277 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf59f9-ccf2-43f1-9a40-13e8f073ee78 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:09.820624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf59f9-ccf2-43f1-9a40-13e8f073ee78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:09.841402 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-d3cf59f9-ccf2-43f1-9a40-13e8f073ee78 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:41:09.842721 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-d3cf59f9-ccf2-43f1-9a40-13e8f073ee78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:09.850958 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1419/2813] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:41:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:41:09.856010 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dab2e2-eb12-4d46-852e-26e3d2aec6b4 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:09.866537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dab2e2-eb12-4d46-852e-26e3d2aec6b4 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:09.871636 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd1638e-65d4-4059-a64d-5f22021c0fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:09.872393 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1638e-65d4-4059-a64d-5f22021c0fc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:09.872857 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1638e-65d4-4059-a64d-5f22021c0fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:09.873148 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1638e-65d4-4059-a64d-5f22021c0fc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:09.873805 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd1638e-65d4-4059-a64d-5f22021c0fc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:41:09.874148 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd1638e-65d4-4059-a64d-5f22021c0fc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:09.876004 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd1638e-65d4-4059-a64d-5f22021c0fc5 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:09.889276 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1395/2814] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:41:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:41:09.898381 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf59f9-ccf2-43f1-9a40-13e8f073ee78 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:09.909577 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf59f9-ccf2-43f1-9a40-13e8f073ee78 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:09.916106 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f511010-8d55-40a6-a9e3-74fc8329260f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:09.916720 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f511010-8d55-40a6-a9e3-74fc8329260f admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:09.917005 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f511010-8d55-40a6-a9e3-74fc8329260f admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:09.918013 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f511010-8d55-40a6-a9e3-74fc8329260f admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:09.918691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f511010-8d55-40a6-a9e3-74fc8329260f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:09.921599 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f511010-8d55-40a6-a9e3-74fc8329260f admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:09.964869 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1420/2815] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:41:09 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:41:09.965908 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd1638e-65d4-4059-a64d-5f22021c0fc5 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:09.975888 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd1638e-65d4-4059-a64d-5f22021c0fc5 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:10.033680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc148edd-c24b-4721-a41d-09260e6b1020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:10.034359 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc148edd-c24b-4721-a41d-09260e6b1020 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:10.034591 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc148edd-c24b-4721-a41d-09260e6b1020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:10.034865 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc148edd-c24b-4721-a41d-09260e6b1020 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:10.035498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc148edd-c24b-4721-a41d-09260e6b1020 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:10.037401 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc148edd-c24b-4721-a41d-09260e6b1020 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:10.057618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1396/2816] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:41:09 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:41:10.061780 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f511010-8d55-40a6-a9e3-74fc8329260f admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:10.072248 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f511010-8d55-40a6-a9e3-74fc8329260f admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:10.077667 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a741b-94e7-4509-b27f-ec9b9e4c90c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:10.078293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a741b-94e7-4509-b27f-ec9b9e4c90c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:10.078571 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a741b-94e7-4509-b27f-ec9b9e4c90c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:10.078843 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a741b-94e7-4509-b27f-ec9b9e4c90c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:10.079492 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306a741b-94e7-4509-b27f-ec9b9e4c90c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:10.081224 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306a741b-94e7-4509-b27f-ec9b9e4c90c8 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:10.091928 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1421/2817] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:41:10.096196 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc148edd-c24b-4721-a41d-09260e6b1020 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:10.106237 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc148edd-c24b-4721-a41d-09260e6b1020 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:10.111158 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a43394-dc1e-498b-b4d8-6ea1201d6045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:10.111751 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a43394-dc1e-498b-b4d8-6ea1201d6045 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:10.112085 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a43394-dc1e-498b-b4d8-6ea1201d6045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:10.112313 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a43394-dc1e-498b-b4d8-6ea1201d6045 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:10.112948 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a43394-dc1e-498b-b4d8-6ea1201d6045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:10.114576 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a43394-dc1e-498b-b4d8-6ea1201d6045 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:10.124912 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1397/2818] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:41:10.128850 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a741b-94e7-4509-b27f-ec9b9e4c90c8 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:10.139259 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a741b-94e7-4509-b27f-ec9b9e4c90c8 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:10.143886 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7162ff-3668-4113-93e7-a74184f88d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:10.144513 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7162ff-3668-4113-93e7-a74184f88d27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:10.144788 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7162ff-3668-4113-93e7-a74184f88d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:10.145058 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7162ff-3668-4113-93e7-a74184f88d27 admin admin] PATH_INFO: `/v3/projects/0e95a355ff1243b6a2cc8f8e6a6590fd/users/99c9076400d045bfb52d063a1d636053/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:10.160152 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7162ff-3668-4113-93e7-a74184f88d27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e95a355ff1243b6a2cc8f8e6a6590fd, user_id=99c9076400d045bfb52d063a1d636053, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:10.162833 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7162ff-3668-4113-93e7-a74184f88d27 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:10.175158 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1422/2819] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:41:10 2022] PUT /identity/v3/projects/0e95a355ff1243b6a2cc8f8e6a6590fd/users/99c9076400d045bfb52d063a1d636053/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:41:10.186506 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39731898-6a8c-4afb-b0dd-63168852c79b None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:10.186782 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39731898-6a8c-4afb-b0dd-63168852c79b None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:10.187073 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39731898-6a8c-4afb-b0dd-63168852c79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:10.214680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-39731898-6a8c-4afb-b0dd-63168852c79b None None] MFA Rules not processed for user `99c9076400d045bfb52d063a1d636053`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:41:10.215915 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-39731898-6a8c-4afb-b0dd-63168852c79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:10.227726 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:41:10.282048 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1398/2820] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:41:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:41:10.286975 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7162ff-3668-4113-93e7-a74184f88d27 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:10.348656 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7162ff-3668-4113-93e7-a74184f88d27 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:10.353343 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ffb6a8-4ed4-428e-84c9-cd9c5351524e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:10.353940 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ffb6a8-4ed4-428e-84c9-cd9c5351524e admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:10.354250 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ffb6a8-4ed4-428e-84c9-cd9c5351524e admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:10.354527 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ffb6a8-4ed4-428e-84c9-cd9c5351524e admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:10.355170 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ffb6a8-4ed4-428e-84c9-cd9c5351524e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:10.357053 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ffb6a8-4ed4-428e-84c9-cd9c5351524e admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:10.362670 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:41:10.385478 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1423/2821] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:41:10 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:41:10.390362 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39731898-6a8c-4afb-b0dd-63168852c79b None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:10.401278 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39731898-6a8c-4afb-b0dd-63168852c79b None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:10.456973 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9545d0-63c0-4da8-9fc7-4934d0662e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:10.457571 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9545d0-63c0-4da8-9fc7-4934d0662e21 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:10.457851 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9545d0-63c0-4da8-9fc7-4934d0662e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:10.458159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9545d0-63c0-4da8-9fc7-4934d0662e21 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:10.458779 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9545d0-63c0-4da8-9fc7-4934d0662e21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:10.460681 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9545d0-63c0-4da8-9fc7-4934d0662e21 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:10.480401 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1399/2822] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:41:10 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:41:10.484644 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ffb6a8-4ed4-428e-84c9-cd9c5351524e admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:10.495022 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ffb6a8-4ed4-428e-84c9-cd9c5351524e admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:10.500121 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad821511-5fa9-40df-a4ff-532407e6cabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:10.500714 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad821511-5fa9-40df-a4ff-532407e6cabe admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:10.501230 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad821511-5fa9-40df-a4ff-532407e6cabe admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:10.501512 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad821511-5fa9-40df-a4ff-532407e6cabe admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:10.502196 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad821511-5fa9-40df-a4ff-532407e6cabe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:10.503778 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad821511-5fa9-40df-a4ff-532407e6cabe admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:10.514554 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1424/2823] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:41:10.518476 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9545d0-63c0-4da8-9fc7-4934d0662e21 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:10.528421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9545d0-63c0-4da8-9fc7-4934d0662e21 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:10.532987 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95bd31b-c3c9-4792-a214-fa059bf31166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:10.533574 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bd31b-c3c9-4792-a214-fa059bf31166 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:10.533853 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bd31b-c3c9-4792-a214-fa059bf31166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:10.534167 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bd31b-c3c9-4792-a214-fa059bf31166 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:10.534802 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95bd31b-c3c9-4792-a214-fa059bf31166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:10.536373 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95bd31b-c3c9-4792-a214-fa059bf31166 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:10.546127 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1400/2824] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:41:10.550273 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad821511-5fa9-40df-a4ff-532407e6cabe admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:10.560734 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad821511-5fa9-40df-a4ff-532407e6cabe admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:10.565536 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db5da65-50a7-4093-bb24-c59d4eccbd43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:10.566217 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5da65-50a7-4093-bb24-c59d4eccbd43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:10.566495 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5da65-50a7-4093-bb24-c59d4eccbd43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:10.566766 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5da65-50a7-4093-bb24-c59d4eccbd43 admin admin] PATH_INFO: `/v3/projects/0b186c440dd34de3b8a73b905550fad8/users/c243beb52e99451ebdbe1a94a4a7be35/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:10.581841 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db5da65-50a7-4093-bb24-c59d4eccbd43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b186c440dd34de3b8a73b905550fad8, user_id=c243beb52e99451ebdbe1a94a4a7be35, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:10.584481 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db5da65-50a7-4093-bb24-c59d4eccbd43 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:10.597258 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1425/2825] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:41:10 2022] PUT /identity/v3/projects/0b186c440dd34de3b8a73b905550fad8/users/c243beb52e99451ebdbe1a94a4a7be35/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:41:10.607340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32aa881-b0b4-4f58-a19b-267503d6d3cb None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:10.607629 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32aa881-b0b4-4f58-a19b-267503d6d3cb None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:10.608159 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32aa881-b0b4-4f58-a19b-267503d6d3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:10.637261 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-c32aa881-b0b4-4f58-a19b-267503d6d3cb None None] MFA Rules not processed for user `c243beb52e99451ebdbe1a94a4a7be35`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:41:10.638432 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-c32aa881-b0b4-4f58-a19b-267503d6d3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:10.700197 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1401/2826] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:41:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:41:10.704707 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db5da65-50a7-4093-bb24-c59d4eccbd43 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:10.765206 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db5da65-50a7-4093-bb24-c59d4eccbd43 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:10.769754 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a51a1-7017-4716-b60e-4ad48e1d65ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:10.770365 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a51a1-7017-4716-b60e-4ad48e1d65ec admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:10.770640 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a51a1-7017-4716-b60e-4ad48e1d65ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:10.770911 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a51a1-7017-4716-b60e-4ad48e1d65ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:10.771524 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a51a1-7017-4716-b60e-4ad48e1d65ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:10.773408 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a51a1-7017-4716-b60e-4ad48e1d65ec admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:10.800454 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1426/2827] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:41:10 2022] POST /identity/v3/projects => generated 375 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:41:10.804442 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32aa881-b0b4-4f58-a19b-267503d6d3cb None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:10.815867 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32aa881-b0b4-4f58-a19b-267503d6d3cb None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:10.833737 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:41:10.874445 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b73c333-cb04-4c0c-a6f5-de6ef61da805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:10.875049 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73c333-cb04-4c0c-a6f5-de6ef61da805 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:10.875328 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73c333-cb04-4c0c-a6f5-de6ef61da805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:10.875602 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73c333-cb04-4c0c-a6f5-de6ef61da805 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:10.876222 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b73c333-cb04-4c0c-a6f5-de6ef61da805 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:10.878165 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b73c333-cb04-4c0c-a6f5-de6ef61da805 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:10.898987 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1402/2828] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:41:10 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:41:10.902927 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a51a1-7017-4716-b60e-4ad48e1d65ec admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:10.912964 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a51a1-7017-4716-b60e-4ad48e1d65ec admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:10.918131 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f9b07-c08e-4443-8953-0874e5c5d56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:10.918717 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f9b07-c08e-4443-8953-0874e5c5d56d admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:10.918991 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f9b07-c08e-4443-8953-0874e5c5d56d admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:10.919259 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f9b07-c08e-4443-8953-0874e5c5d56d admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:10.919886 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9f9b07-c08e-4443-8953-0874e5c5d56d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:10.921455 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9f9b07-c08e-4443-8953-0874e5c5d56d admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:10.933448 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1427/2829] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:41:10.937140 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b73c333-cb04-4c0c-a6f5-de6ef61da805 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:10.946912 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b73c333-cb04-4c0c-a6f5-de6ef61da805 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:10.951486 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb1ea5-f8b7-47c3-b9d2-4bdb9ee326e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:10.952121 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb1ea5-f8b7-47c3-b9d2-4bdb9ee326e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:10.952404 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb1ea5-f8b7-47c3-b9d2-4bdb9ee326e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:10.952682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb1ea5-f8b7-47c3-b9d2-4bdb9ee326e6 admin admin] PATH_INFO: `/v3/projects/75bbe9a03067423abba5bc168be47ace/users/f6e81617d2664ba9836d08fc218059d7/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:10.968556 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb1ea5-f8b7-47c3-b9d2-4bdb9ee326e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75bbe9a03067423abba5bc168be47ace, user_id=f6e81617d2664ba9836d08fc218059d7, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:10.971470 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb1ea5-f8b7-47c3-b9d2-4bdb9ee326e6 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:10.985850 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1403/2830] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:41:10 2022] PUT /identity/v3/projects/75bbe9a03067423abba5bc168be47ace/users/f6e81617d2664ba9836d08fc218059d7/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:41:10.989700 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f9b07-c08e-4443-8953-0874e5c5d56d admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:11.010199 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f9b07-c08e-4443-8953-0874e5c5d56d admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:11.061435 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e99a335-33ad-4f9b-ba53-17e49a83e2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:11.062058 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99a335-33ad-4f9b-ba53-17e49a83e2ee admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:11.062410 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99a335-33ad-4f9b-ba53-17e49a83e2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:11.062680 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99a335-33ad-4f9b-ba53-17e49a83e2ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:11.063306 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e99a335-33ad-4f9b-ba53-17e49a83e2ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:11.064890 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e99a335-33ad-4f9b-ba53-17e49a83e2ee admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:11.076398 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1428/2831] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:41:11.080751 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb1ea5-f8b7-47c3-b9d2-4bdb9ee326e6 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:11.091710 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb1ea5-f8b7-47c3-b9d2-4bdb9ee326e6 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:11.096520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde92980-baca-4604-bc37-0fd04e796f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:11.097182 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde92980-baca-4604-bc37-0fd04e796f95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:11.097456 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde92980-baca-4604-bc37-0fd04e796f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:11.097728 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde92980-baca-4604-bc37-0fd04e796f95 admin admin] PATH_INFO: `/v3/projects/75bbe9a03067423abba5bc168be47ace/users/f6e81617d2664ba9836d08fc218059d7/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:11.099705 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde92980-baca-4604-bc37-0fd04e796f95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75bbe9a03067423abba5bc168be47ace, user_id=f6e81617d2664ba9836d08fc218059d7, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:11.102364 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde92980-baca-4604-bc37-0fd04e796f95 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:11.113463 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1404/2832] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:41:11 2022] PUT /identity/v3/projects/75bbe9a03067423abba5bc168be47ace/users/f6e81617d2664ba9836d08fc218059d7/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:41:11.124563 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e53935-2d80-432c-9ee1-db1b3a63417a None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:11.124906 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e53935-2d80-432c-9ee1-db1b3a63417a None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:11.125151 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e53935-2d80-432c-9ee1-db1b3a63417a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:11.162772 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-53e53935-2d80-432c-9ee1-db1b3a63417a None None] MFA Rules not processed for user `f6e81617d2664ba9836d08fc218059d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:41:11.163999 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-53e53935-2d80-432c-9ee1-db1b3a63417a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:11.174990 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:41:11.236823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1429/2833] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:41:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:41:11.248973 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde92980-baca-4604-bc37-0fd04e796f95 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:11.250695 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-bde92980-baca-4604-bc37-0fd04e796f95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:11.308841 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde92980-baca-4604-bc37-0fd04e796f95 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:11.312447 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251e0e0-23c0-4114-8252-aa92e9837f49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:11.313018 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251e0e0-23c0-4114-8252-aa92e9837f49 service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:11.313299 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251e0e0-23c0-4114-8252-aa92e9837f49 service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:11.313565 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251e0e0-23c0-4114-8252-aa92e9837f49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:11.369632 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1251e0e0-23c0-4114-8252-aa92e9837f49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:11.371112 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1251e0e0-23c0-4114-8252-aa92e9837f49 service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:11.379978 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1405/2834] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:41:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:41:14.979303 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e53935-2d80-432c-9ee1-db1b3a63417a None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:14.989277 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e53935-2d80-432c-9ee1-db1b3a63417a None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:14.992869 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd27b7e-904e-463b-af26-606652a84921 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:14.993464 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd27b7e-904e-463b-af26-606652a84921 service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:14.993745 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd27b7e-904e-463b-af26-606652a84921 service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:14.994035 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd27b7e-904e-463b-af26-606652a84921 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:15.049552 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd27b7e-904e-463b-af26-606652a84921 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:15.051063 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd27b7e-904e-463b-af26-606652a84921 service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:15.060361 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1430/2835] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:41:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:41:15.602774 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251e0e0-23c0-4114-8252-aa92e9837f49 service cinder] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:15.604211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-1251e0e0-23c0-4114-8252-aa92e9837f49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:15.617199 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251e0e0-23c0-4114-8252-aa92e9837f49 service cinder] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:15.621869 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7a05c-06a0-4a06-b5e5-8506d7319752 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:15.622494 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7a05c-06a0-4a06-b5e5-8506d7319752 service cinder] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:15.622769 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7a05c-06a0-4a06-b5e5-8506d7319752 service cinder] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:15.623035 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7a05c-06a0-4a06-b5e5-8506d7319752 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:15.633150 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d7a05c-06a0-4a06-b5e5-8506d7319752 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:15.634612 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d7a05c-06a0-4a06-b5e5-8506d7319752 service cinder] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:15.654142 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1406/2836] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:41:15 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:41:19.537277 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd27b7e-904e-463b-af26-606652a84921 service cinder] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:19.599462 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd27b7e-904e-463b-af26-606652a84921 service cinder] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:19.604306 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531fa0c0-017b-4fe1-9539-fd5c18e612b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:19.604938 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531fa0c0-017b-4fe1-9539-fd5c18e612b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:19.605219 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531fa0c0-017b-4fe1-9539-fd5c18e612b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:19.605490 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531fa0c0-017b-4fe1-9539-fd5c18e612b0 admin admin] PATH_INFO: `/v3/users/99c9076400d045bfb52d063a1d636053` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:19.606634 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531fa0c0-017b-4fe1-9539-fd5c18e612b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99c9076400d045bfb52d063a1d636053)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:19.608879 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531fa0c0-017b-4fe1-9539-fd5c18e612b0 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:19.655655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-531fa0c0-017b-4fe1-9539-fd5c18e612b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99c9076400d045bfb52d063a1d636053'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:19.656292 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-531fa0c0-017b-4fe1-9539-fd5c18e612b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99c9076400d045bfb52d063a1d636053'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:19.657496 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:41:19.665050 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-531fa0c0-017b-4fe1-9539-fd5c18e612b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99c9076400d045bfb52d063a1d636053'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:19.676660 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-531fa0c0-017b-4fe1-9539-fd5c18e612b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99c9076400d045bfb52d063a1d636053'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:19.677310 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-531fa0c0-017b-4fe1-9539-fd5c18e612b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99c9076400d045bfb52d063a1d636053'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:19.690953 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1431/2837] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:41:19 2022] DELETE /identity/v3/users/99c9076400d045bfb52d063a1d636053 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:41:19.695370 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15876a-481f-49bb-919e-860eedba2836 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:19.695876 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15876a-481f-49bb-919e-860eedba2836 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:19.695876 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15876a-481f-49bb-919e-860eedba2836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:19.715704 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-4c15876a-481f-49bb-919e-860eedba2836 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:41:19.716893 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-4c15876a-481f-49bb-919e-860eedba2836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:19.726269 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:41:19.776744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1407/2838] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:41:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:41:19.786410 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531fa0c0-017b-4fe1-9539-fd5c18e612b0 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:19.787786 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-531fa0c0-017b-4fe1-9539-fd5c18e612b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:19.846801 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531fa0c0-017b-4fe1-9539-fd5c18e612b0 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:19.850440 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f3476-f9c7-4b67-a8cf-b9a1229c75ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:19.851037 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f3476-f9c7-4b67-a8cf-b9a1229c75ad service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:19.851315 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f3476-f9c7-4b67-a8cf-b9a1229c75ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:19.851589 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f3476-f9c7-4b67-a8cf-b9a1229c75ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:19.860839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f3476-f9c7-4b67-a8cf-b9a1229c75ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:19.862224 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f3476-f9c7-4b67-a8cf-b9a1229c75ad service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:19.871643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1432/2839] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:41:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:41:20.134414 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15876a-481f-49bb-919e-860eedba2836 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:20.136603 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-4c15876a-481f-49bb-919e-860eedba2836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:20.150719 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15876a-481f-49bb-919e-860eedba2836 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:20.156322 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe6c03-e88c-422e-8c01-25bd5d3cf27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:20.157211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe6c03-e88c-422e-8c01-25bd5d3cf27d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:20.157564 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe6c03-e88c-422e-8c01-25bd5d3cf27d admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:20.157904 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe6c03-e88c-422e-8c01-25bd5d3cf27d admin admin] PATH_INFO: `/v3/projects/0e95a355ff1243b6a2cc8f8e6a6590fd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:20.159260 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe6c03-e88c-422e-8c01-25bd5d3cf27d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e95a355ff1243b6a2cc8f8e6a6590fd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:20.161545 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe6c03-e88c-422e-8c01-25bd5d3cf27d admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:20.213442 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-b1fe6c03-e88c-422e-8c01-25bd5d3cf27d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e95a355ff1243b6a2cc8f8e6a6590fd', 'request_id': 'req-b1fe6c03-e88c-422e-8c01-25bd5d3cf27d'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:20.219839 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-b1fe6c03-e88c-422e-8c01-25bd5d3cf27d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e95a355ff1243b6a2cc8f8e6a6590fd', 'request_id': 'req-b1fe6c03-e88c-422e-8c01-25bd5d3cf27d'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:20.225432 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-b1fe6c03-e88c-422e-8c01-25bd5d3cf27d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e95a355ff1243b6a2cc8f8e6a6590fd', 'request_id': 'req-b1fe6c03-e88c-422e-8c01-25bd5d3cf27d'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:20.238622 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-b1fe6c03-e88c-422e-8c01-25bd5d3cf27d admin admin] The token cache is being invalidate because project 0e95a355ff1243b6a2cc8f8e6a6590fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:41:20.239025 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-b1fe6c03-e88c-422e-8c01-25bd5d3cf27d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:20.239674 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-b1fe6c03-e88c-422e-8c01-25bd5d3cf27d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:20.241290 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1408/2840] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:41:20 2022] DELETE /identity/v3/projects/0e95a355ff1243b6a2cc8f8e6a6590fd => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:41:20.246228 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f3476-f9c7-4b67-a8cf-b9a1229c75ad service neutron] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:20.257345 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f3476-f9c7-4b67-a8cf-b9a1229c75ad service neutron] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:20.314392 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c947e80-835b-4613-b0e5-e5bb80c01675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:20.315015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c947e80-835b-4613-b0e5-e5bb80c01675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:20.315293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c947e80-835b-4613-b0e5-e5bb80c01675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:20.315567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c947e80-835b-4613-b0e5-e5bb80c01675 admin admin] PATH_INFO: `/v3/users/c243beb52e99451ebdbe1a94a4a7be35` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:20.316671 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c947e80-835b-4613-b0e5-e5bb80c01675 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c243beb52e99451ebdbe1a94a4a7be35)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:20.318633 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c947e80-835b-4613-b0e5-e5bb80c01675 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:20.365009 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7c947e80-835b-4613-b0e5-e5bb80c01675 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c243beb52e99451ebdbe1a94a4a7be35'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:20.365649 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7c947e80-835b-4613-b0e5-e5bb80c01675 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c243beb52e99451ebdbe1a94a4a7be35'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:20.373083 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7c947e80-835b-4613-b0e5-e5bb80c01675 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c243beb52e99451ebdbe1a94a4a7be35'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:20.383424 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:41:20.386495 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7c947e80-835b-4613-b0e5-e5bb80c01675 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c243beb52e99451ebdbe1a94a4a7be35'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:20.387625 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-7c947e80-835b-4613-b0e5-e5bb80c01675 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c243beb52e99451ebdbe1a94a4a7be35'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:20.401487 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1433/2841] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:41:20 2022] DELETE /identity/v3/users/c243beb52e99451ebdbe1a94a4a7be35 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:41:20.669673 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe6c03-e88c-422e-8c01-25bd5d3cf27d admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:20.671125 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-b1fe6c03-e88c-422e-8c01-25bd5d3cf27d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:20.686447 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe6c03-e88c-422e-8c01-25bd5d3cf27d admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:20.744208 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc1262e-bd9d-47f7-a28d-ccb5ac02df65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:20.744977 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1262e-bd9d-47f7-a28d-ccb5ac02df65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:20.745299 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1262e-bd9d-47f7-a28d-ccb5ac02df65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:20.745574 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1262e-bd9d-47f7-a28d-ccb5ac02df65 admin admin] PATH_INFO: `/v3/projects/0b186c440dd34de3b8a73b905550fad8` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:20.746708 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc1262e-bd9d-47f7-a28d-ccb5ac02df65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b186c440dd34de3b8a73b905550fad8)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:20.748601 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc1262e-bd9d-47f7-a28d-ccb5ac02df65 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:20.788696 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:41:20.798968 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-5bc1262e-bd9d-47f7-a28d-ccb5ac02df65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b186c440dd34de3b8a73b905550fad8', 'request_id': 'req-5bc1262e-bd9d-47f7-a28d-ccb5ac02df65'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:20.805508 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-5bc1262e-bd9d-47f7-a28d-ccb5ac02df65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b186c440dd34de3b8a73b905550fad8', 'request_id': 'req-5bc1262e-bd9d-47f7-a28d-ccb5ac02df65'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:20.811191 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-5bc1262e-bd9d-47f7-a28d-ccb5ac02df65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b186c440dd34de3b8a73b905550fad8', 'request_id': 'req-5bc1262e-bd9d-47f7-a28d-ccb5ac02df65'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:20.822737 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-5bc1262e-bd9d-47f7-a28d-ccb5ac02df65 admin admin] The token cache is being invalidate because project 0b186c440dd34de3b8a73b905550fad8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:41:20.823025 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-5bc1262e-bd9d-47f7-a28d-ccb5ac02df65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:20.824190 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-5bc1262e-bd9d-47f7-a28d-ccb5ac02df65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:20.825686 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1409/2842] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:41:20 2022] DELETE /identity/v3/projects/0b186c440dd34de3b8a73b905550fad8 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:41:20.829681 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c947e80-835b-4613-b0e5-e5bb80c01675 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:20.831116 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-7c947e80-835b-4613-b0e5-e5bb80c01675 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:20.844297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c947e80-835b-4613-b0e5-e5bb80c01675 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:20.899886 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13223cc3-0fad-4f48-87ef-fbbdadedd522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:20.900505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13223cc3-0fad-4f48-87ef-fbbdadedd522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:20.900783 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13223cc3-0fad-4f48-87ef-fbbdadedd522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:20.901048 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13223cc3-0fad-4f48-87ef-fbbdadedd522 admin admin] PATH_INFO: `/v3/users/f6e81617d2664ba9836d08fc218059d7` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:20.902185 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13223cc3-0fad-4f48-87ef-fbbdadedd522 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6e81617d2664ba9836d08fc218059d7)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:20.904087 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13223cc3-0fad-4f48-87ef-fbbdadedd522 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:20.949209 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-13223cc3-0fad-4f48-87ef-fbbdadedd522 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6e81617d2664ba9836d08fc218059d7'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:20.949783 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-13223cc3-0fad-4f48-87ef-fbbdadedd522 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6e81617d2664ba9836d08fc218059d7'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:20.956684 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-13223cc3-0fad-4f48-87ef-fbbdadedd522 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6e81617d2664ba9836d08fc218059d7'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:20.968336 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-13223cc3-0fad-4f48-87ef-fbbdadedd522 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6e81617d2664ba9836d08fc218059d7'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:20.968967 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-13223cc3-0fad-4f48-87ef-fbbdadedd522 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6e81617d2664ba9836d08fc218059d7'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:20.981777 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1434/2843] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:41:20 2022] DELETE /identity/v3/users/f6e81617d2664ba9836d08fc218059d7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:41:21.253301 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc1262e-bd9d-47f7-a28d-ccb5ac02df65 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:21.254788 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-5bc1262e-bd9d-47f7-a28d-ccb5ac02df65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:21.269934 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc1262e-bd9d-47f7-a28d-ccb5ac02df65 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:21.331479 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18900f69-9d5b-47cc-b178-071a3a0099ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:21.332152 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18900f69-9d5b-47cc-b178-071a3a0099ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:21.332434 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18900f69-9d5b-47cc-b178-071a3a0099ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:21.332709 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18900f69-9d5b-47cc-b178-071a3a0099ab admin admin] PATH_INFO: `/v3/projects/75bbe9a03067423abba5bc168be47ace` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:21.333857 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18900f69-9d5b-47cc-b178-071a3a0099ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=75bbe9a03067423abba5bc168be47ace)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:21.335868 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18900f69-9d5b-47cc-b178-071a3a0099ab admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:21.388707 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-18900f69-9d5b-47cc-b178-071a3a0099ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75bbe9a03067423abba5bc168be47ace', 'request_id': 'req-18900f69-9d5b-47cc-b178-071a3a0099ab'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:21.395327 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-18900f69-9d5b-47cc-b178-071a3a0099ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75bbe9a03067423abba5bc168be47ace', 'request_id': 'req-18900f69-9d5b-47cc-b178-071a3a0099ab'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:21.404582 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-18900f69-9d5b-47cc-b178-071a3a0099ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75bbe9a03067423abba5bc168be47ace', 'request_id': 'req-18900f69-9d5b-47cc-b178-071a3a0099ab'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:21.416771 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-18900f69-9d5b-47cc-b178-071a3a0099ab admin admin] The token cache is being invalidate because project 75bbe9a03067423abba5bc168be47ace was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:41:21.417092 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-18900f69-9d5b-47cc-b178-071a3a0099ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:21.417716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-18900f69-9d5b-47cc-b178-071a3a0099ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:21.419286 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1410/2844] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:41:21 2022] DELETE /identity/v3/projects/75bbe9a03067423abba5bc168be47ace => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:41:21.424536 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280c0a8-85a2-4bb8-8c4c-c1e05d37a04f None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:21.424822 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280c0a8-85a2-4bb8-8c4c-c1e05d37a04f None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:21.425096 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280c0a8-85a2-4bb8-8c4c-c1e05d37a04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:21.446172 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-8280c0a8-85a2-4bb8-8c4c-c1e05d37a04f None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:41:21.447369 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-8280c0a8-85a2-4bb8-8c4c-c1e05d37a04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:21.509501 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1435/2845] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:41:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:41:21.520400 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342bfb9b-ce11-451b-9a91-392de22d218d None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:21.520680 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342bfb9b-ce11-451b-9a91-392de22d218d None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:21.521015 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342bfb9b-ce11-451b-9a91-392de22d218d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:21.545471 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-342bfb9b-ce11-451b-9a91-392de22d218d None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:41:21.545471 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-342bfb9b-ce11-451b-9a91-392de22d218d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:21.552654 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1411/2846] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:41:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:41:21.558956 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280c0a8-85a2-4bb8-8c4c-c1e05d37a04f None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:21.569257 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280c0a8-85a2-4bb8-8c4c-c1e05d37a04f None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:21.574625 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e028d15-fc96-4ef9-becc-8275af8a50f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:21.575236 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e028d15-fc96-4ef9-becc-8275af8a50f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:21.575520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e028d15-fc96-4ef9-becc-8275af8a50f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:21.575798 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e028d15-fc96-4ef9-becc-8275af8a50f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:21.576579 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e028d15-fc96-4ef9-becc-8275af8a50f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:41:21.576886 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e028d15-fc96-4ef9-becc-8275af8a50f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:21.578567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e028d15-fc96-4ef9-becc-8275af8a50f9 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:21.580239 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:41:21.593876 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1436/2847] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:41:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:41:21.598339 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342bfb9b-ce11-451b-9a91-392de22d218d None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:21.607955 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342bfb9b-ce11-451b-9a91-392de22d218d None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:21.613078 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb935da-8400-4fb8-8ee8-51de9893282e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:21.613774 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb935da-8400-4fb8-8ee8-51de9893282e admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:21.613987 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb935da-8400-4fb8-8ee8-51de9893282e admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:21.614280 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb935da-8400-4fb8-8ee8-51de9893282e admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:21.614920 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb935da-8400-4fb8-8ee8-51de9893282e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:21.616825 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb935da-8400-4fb8-8ee8-51de9893282e admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:21.648189 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1412/2848] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:41:21 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:41:21.652404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e028d15-fc96-4ef9-becc-8275af8a50f9 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:21.662384 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e028d15-fc96-4ef9-becc-8275af8a50f9 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:21.716909 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662d6b8-e3f0-4dd4-bd34-634a6a140f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:21.717505 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662d6b8-e3f0-4dd4-bd34-634a6a140f43 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:21.717780 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662d6b8-e3f0-4dd4-bd34-634a6a140f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:21.718073 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662d6b8-e3f0-4dd4-bd34-634a6a140f43 admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:21.718687 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662d6b8-e3f0-4dd4-bd34-634a6a140f43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:21.720571 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662d6b8-e3f0-4dd4-bd34-634a6a140f43 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:21.741322 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1437/2849] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:41:21 2022] POST /identity/v3/users => generated 287 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:41:21.745959 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb935da-8400-4fb8-8ee8-51de9893282e admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:21.756245 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb935da-8400-4fb8-8ee8-51de9893282e admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:21.761507 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a084d1-b73e-4c43-88c3-ee76d53f658d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:21.762042 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a084d1-b73e-4c43-88c3-ee76d53f658d admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:21.762340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a084d1-b73e-4c43-88c3-ee76d53f658d admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:21.762615 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a084d1-b73e-4c43-88c3-ee76d53f658d admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:21.763251 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a084d1-b73e-4c43-88c3-ee76d53f658d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:21.764871 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a084d1-b73e-4c43-88c3-ee76d53f658d admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:21.774674 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1413/2850] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:41:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:41:21.778502 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662d6b8-e3f0-4dd4-bd34-634a6a140f43 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:21.788511 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662d6b8-e3f0-4dd4-bd34-634a6a140f43 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:21.793197 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3c503d-91d6-4692-b775-f9d14db2f8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:21.793798 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c503d-91d6-4692-b775-f9d14db2f8ac admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:21.794108 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c503d-91d6-4692-b775-f9d14db2f8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:21.794388 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c503d-91d6-4692-b775-f9d14db2f8ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:21.795057 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3c503d-91d6-4692-b775-f9d14db2f8ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:21.796655 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3c503d-91d6-4692-b775-f9d14db2f8ac admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:21.808166 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1438/2851] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:41:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:41:21.812623 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a084d1-b73e-4c43-88c3-ee76d53f658d admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:21.822804 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a084d1-b73e-4c43-88c3-ee76d53f658d admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:21.827983 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2951a980-32c5-4278-90ed-12246fc196dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:21.828646 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951a980-32c5-4278-90ed-12246fc196dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:21.828924 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951a980-32c5-4278-90ed-12246fc196dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:21.829196 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951a980-32c5-4278-90ed-12246fc196dc admin admin] PATH_INFO: `/v3/projects/b35e6a97e32b426daab0ba663323e8a8/users/c1dcec4c0b364d7caa37af361da31b18/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:21.848211 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2951a980-32c5-4278-90ed-12246fc196dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b35e6a97e32b426daab0ba663323e8a8, user_id=c1dcec4c0b364d7caa37af361da31b18, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:21.850898 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2951a980-32c5-4278-90ed-12246fc196dc admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:21.853973 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:41:21.866315 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1414/2852] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:41:21 2022] PUT /identity/v3/projects/b35e6a97e32b426daab0ba663323e8a8/users/c1dcec4c0b364d7caa37af361da31b18/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:41:21.872053 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a1305-546a-447a-88e5-94a77cf14791 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:21.872518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a1305-546a-447a-88e5-94a77cf14791 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:21.872617 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a1305-546a-447a-88e5-94a77cf14791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:21.894712 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-595a1305-546a-447a-88e5-94a77cf14791 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:41:21.895960 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-595a1305-546a-447a-88e5-94a77cf14791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:21.954665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1439/2853] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:41:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:41:21.964346 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2951a980-32c5-4278-90ed-12246fc196dc admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:21.966007 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-2951a980-32c5-4278-90ed-12246fc196dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:22.023767 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2951a980-32c5-4278-90ed-12246fc196dc admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:22.027288 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3ac4f-ac1b-48df-abce-a6396a5365d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:22.027878 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3ac4f-ac1b-48df-abce-a6396a5365d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:22.028154 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3ac4f-ac1b-48df-abce-a6396a5365d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:22.028421 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3ac4f-ac1b-48df-abce-a6396a5365d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:22.037222 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c3ac4f-ac1b-48df-abce-a6396a5365d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:22.038668 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c3ac4f-ac1b-48df-abce-a6396a5365d8 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:22.048490 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1415/2854] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:41:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:41:25.427612 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a1305-546a-447a-88e5-94a77cf14791 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:25.432847 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-595a1305-546a-447a-88e5-94a77cf14791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:25.504011 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3ac4f-ac1b-48df-abce-a6396a5365d8 service neutron] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:25.505655 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-54c3ac4f-ac1b-48df-abce-a6396a5365d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:25.524117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3ac4f-ac1b-48df-abce-a6396a5365d8 service neutron] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:25.532251 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1324c2-690a-4dc3-96c8-c3184f0fc80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:25.533937 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1324c2-690a-4dc3-96c8-c3184f0fc80a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:25.535658 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1324c2-690a-4dc3-96c8-c3184f0fc80a admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:25.541798 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1324c2-690a-4dc3-96c8-c3184f0fc80a admin admin] PATH_INFO: `/v3/users/57fa5ee68e6f48e982c312e6bb3a6ce6` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:25.543388 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1324c2-690a-4dc3-96c8-c3184f0fc80a admin admin] RBAC: Authorizing `identity:delete_user(user_id=57fa5ee68e6f48e982c312e6bb3a6ce6)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:25.545598 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1324c2-690a-4dc3-96c8-c3184f0fc80a admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:25.601732 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a1305-546a-447a-88e5-94a77cf14791 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:25.608000 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dca6c0-a2e2-434f-af57-2199f720b155 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84b02e7351914cabb67462454b328d05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:25.608611 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dca6c0-a2e2-434f-af57-2199f720b155 service placement] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:25.608895 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dca6c0-a2e2-434f-af57-2199f720b155 service placement] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:25.609171 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dca6c0-a2e2-434f-af57-2199f720b155 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:25.611392 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-17dca6c0-a2e2-434f-af57-2199f720b155 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:25.619666 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-9a1324c2-690a-4dc3-96c8-c3184f0fc80a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57fa5ee68e6f48e982c312e6bb3a6ce6'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:25.620574 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-9a1324c2-690a-4dc3-96c8-c3184f0fc80a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57fa5ee68e6f48e982c312e6bb3a6ce6'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:25.621789 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-9a1324c2-690a-4dc3-96c8-c3184f0fc80a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57fa5ee68e6f48e982c312e6bb3a6ce6'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:25.631107 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dca6c0-a2e2-434f-af57-2199f720b155 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:25.632717 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dca6c0-a2e2-434f-af57-2199f720b155 service placement] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:25.634072 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-9a1324c2-690a-4dc3-96c8-c3184f0fc80a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57fa5ee68e6f48e982c312e6bb3a6ce6'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:25.643520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1440/2855] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:41:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:41:25.657051 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-9a1324c2-690a-4dc3-96c8-c3184f0fc80a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57fa5ee68e6f48e982c312e6bb3a6ce6'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:25.675639 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1416/2856] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:41:25 2022] DELETE /identity/v3/users/57fa5ee68e6f48e982c312e6bb3a6ce6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:41:25.905449 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dca6c0-a2e2-434f-af57-2199f720b155 service placement] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:25.918673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:41:25.925897 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dca6c0-a2e2-434f-af57-2199f720b155 service placement] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:26.025704 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1872a63-0759-4d75-b856-fee63223b037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:26.028167 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1872a63-0759-4d75-b856-fee63223b037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:26.029211 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1872a63-0759-4d75-b856-fee63223b037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:26.029711 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1872a63-0759-4d75-b856-fee63223b037 admin admin] PATH_INFO: `/v3/projects/3e71ac778bf24f779d6d4c8c5de7e528` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:26.031219 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1872a63-0759-4d75-b856-fee63223b037 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e71ac778bf24f779d6d4c8c5de7e528)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:26.034954 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1872a63-0759-4d75-b856-fee63223b037 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:26.123755 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-a1872a63-0759-4d75-b856-fee63223b037 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e71ac778bf24f779d6d4c8c5de7e528', 'request_id': 'req-a1872a63-0759-4d75-b856-fee63223b037'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:26.131686 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-a1872a63-0759-4d75-b856-fee63223b037 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e71ac778bf24f779d6d4c8c5de7e528', 'request_id': 'req-a1872a63-0759-4d75-b856-fee63223b037'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:26.143700 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-a1872a63-0759-4d75-b856-fee63223b037 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e71ac778bf24f779d6d4c8c5de7e528', 'request_id': 'req-a1872a63-0759-4d75-b856-fee63223b037'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:26.159058 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-a1872a63-0759-4d75-b856-fee63223b037 admin admin] The token cache is being invalidate because project 3e71ac778bf24f779d6d4c8c5de7e528 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81420) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:41:26.159638 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-a1872a63-0759-4d75-b856-fee63223b037 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:26.160731 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-a1872a63-0759-4d75-b856-fee63223b037 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:26.163413 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1441/2857] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:41:25 2022] DELETE /identity/v3/projects/3e71ac778bf24f779d6d4c8c5de7e528 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:41:26.168289 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83371486-e6dd-42d1-8604-08a81229f002 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:26.168857 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83371486-e6dd-42d1-8604-08a81229f002 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:26.169202 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83371486-e6dd-42d1-8604-08a81229f002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:26.208929 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-83371486-e6dd-42d1-8604-08a81229f002 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:41:26.210355 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-83371486-e6dd-42d1-8604-08a81229f002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:26.236170 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362ac6a-48b6-4c6b-8be3-a678ad403c21 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:26.236450 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362ac6a-48b6-4c6b-8be3-a678ad403c21 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:26.236738 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362ac6a-48b6-4c6b-8be3-a678ad403c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:26.273357 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-0362ac6a-48b6-4c6b-8be3-a678ad403c21 None None] MFA Rules not processed for user `c1dcec4c0b364d7caa37af361da31b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:41:26.274894 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-0362ac6a-48b6-4c6b-8be3-a678ad403c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:26.293476 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1417/2858] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:41:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:41:26.303423 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880278c-ccd4-48a9-97b7-dda0ea02d5ed None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:26.303716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880278c-ccd4-48a9-97b7-dda0ea02d5ed None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:26.303991 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880278c-ccd4-48a9-97b7-dda0ea02d5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:26.333109 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-3880278c-ccd4-48a9-97b7-dda0ea02d5ed None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:41:26.335046 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-3880278c-ccd4-48a9-97b7-dda0ea02d5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:26.346369 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1418/2859] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:41:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:41:26.355220 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880278c-ccd4-48a9-97b7-dda0ea02d5ed None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:26.361682 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:41:26.366818 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1442/2860] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:41:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:41:26.368983 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880278c-ccd4-48a9-97b7-dda0ea02d5ed None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:26.373365 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362ac6a-48b6-4c6b-8be3-a678ad403c21 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:26.374967 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-0362ac6a-48b6-4c6b-8be3-a678ad403c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:26.376122 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a192a2-487a-44bb-9bda-a9c321b22187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:26.376929 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a192a2-487a-44bb-9bda-a9c321b22187 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:26.377459 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a192a2-487a-44bb-9bda-a9c321b22187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:26.377898 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a192a2-487a-44bb-9bda-a9c321b22187 admin admin] PATH_INFO: `/v3/domains` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:26.378775 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a192a2-487a-44bb-9bda-a9c321b22187 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:41:26.379233 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a192a2-487a-44bb-9bda-a9c321b22187 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:26.381270 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a192a2-487a-44bb-9bda-a9c321b22187 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:26.392212 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362ac6a-48b6-4c6b-8be3-a678ad403c21 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:26.396717 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1419/2861] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:41:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 10:41:26.401998 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a192a2-487a-44bb-9bda-a9c321b22187 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:26.402189 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6819af30-b682-44da-82cc-d30e57a86056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:26.406297 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819af30-b682-44da-82cc-d30e57a86056 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:26.406650 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819af30-b682-44da-82cc-d30e57a86056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:26.406918 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819af30-b682-44da-82cc-d30e57a86056 admin admin] PATH_INFO: `/v3/projects` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:26.407588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6819af30-b682-44da-82cc-d30e57a86056 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:26.409529 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6819af30-b682-44da-82cc-d30e57a86056 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:26.412635 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a192a2-487a-44bb-9bda-a9c321b22187 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:26.418732 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395ea8d-cf21-42ff-8912-146996b27003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:26.419020 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395ea8d-cf21-42ff-8912-146996b27003 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:26.419233 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395ea8d-cf21-42ff-8912-146996b27003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:26.419508 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395ea8d-cf21-42ff-8912-146996b27003 admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:26.420346 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5395ea8d-cf21-42ff-8912-146996b27003 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:26.422369 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5395ea8d-cf21-42ff-8912-146996b27003 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:26.451438 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:41:26.455614 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1420/2862] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:41:26 2022] POST /identity/v3/projects => generated 397 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:41:26.463240 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395ea8d-cf21-42ff-8912-146996b27003 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:26.468175 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1443/2863] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:41:26 2022] POST /identity/v3/projects => generated 371 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:41:26.469146 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6819af30-b682-44da-82cc-d30e57a86056 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:26.548286 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6819af30-b682-44da-82cc-d30e57a86056 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:26.550223 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395ea8d-cf21-42ff-8912-146996b27003 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:26.555485 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bfbd4f-c0ff-4eaf-92a1-07358bf92815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:26.556126 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfbd4f-c0ff-4eaf-92a1-07358bf92815 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:26.556416 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0010c8-3716-4ce4-8ac2-daf13a885797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:26.556552 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfbd4f-c0ff-4eaf-92a1-07358bf92815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:26.556738 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfbd4f-c0ff-4eaf-92a1-07358bf92815 admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:26.557355 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bfbd4f-c0ff-4eaf-92a1-07358bf92815 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:26.558673 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0010c8-3716-4ce4-8ac2-daf13a885797 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:26.559145 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0010c8-3716-4ce4-8ac2-daf13a885797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:26.559414 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0010c8-3716-4ce4-8ac2-daf13a885797 admin admin] PATH_INFO: `/v3/users` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:26.559994 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bfbd4f-c0ff-4eaf-92a1-07358bf92815 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:26.560128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0010c8-3716-4ce4-8ac2-daf13a885797 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:26.562040 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0010c8-3716-4ce4-8ac2-daf13a885797 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:26.589832 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1444/2864] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:41:26 2022] POST /identity/v3/users => generated 292 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:41:26.591213 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bfbd4f-c0ff-4eaf-92a1-07358bf92815 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:26.595333 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1421/2865] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:41:26 2022] POST /identity/v3/users => generated 299 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:41:26.599561 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0010c8-3716-4ce4-8ac2-daf13a885797 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:26.602592 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bfbd4f-c0ff-4eaf-92a1-07358bf92815 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:26.613080 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0010c8-3716-4ce4-8ac2-daf13a885797 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:26.619178 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1f489-d392-4efa-b441-f7cf68358701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:26.619796 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1f489-d392-4efa-b441-f7cf68358701 admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:26.620105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1f489-d392-4efa-b441-f7cf68358701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:26.620927 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1f489-d392-4efa-b441-f7cf68358701 admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:26.621600 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b1f489-d392-4efa-b441-f7cf68358701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:26.623213 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b1f489-d392-4efa-b441-f7cf68358701 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:26.634386 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af38f3-857a-447e-91a1-9138fda0e1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:26.635022 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af38f3-857a-447e-91a1-9138fda0e1f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:26.635293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af38f3-857a-447e-91a1-9138fda0e1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:26.635620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af38f3-857a-447e-91a1-9138fda0e1f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:26.636305 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23af38f3-857a-447e-91a1-9138fda0e1f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:26.638065 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23af38f3-857a-447e-91a1-9138fda0e1f7 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:26.641120 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1445/2866] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:41:26.645218 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1f489-d392-4efa-b441-f7cf68358701 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:26.653677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1422/2867] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 10:41:26.654503 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af38f3-857a-447e-91a1-9138fda0e1f7 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:26.656848 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1f489-d392-4efa-b441-f7cf68358701 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:26.662580 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc5be7-7faf-4415-b9bf-3fd99043c6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:26.663675 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc5be7-7faf-4415-b9bf-3fd99043c6dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:26.664030 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc5be7-7faf-4415-b9bf-3fd99043c6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:26.664426 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc5be7-7faf-4415-b9bf-3fd99043c6dc admin admin] PATH_INFO: `/v3/projects/967192ce42d64f9b886a6e5d7b194283/users/677da7e0e2ed4e72a66f0ed608177bc2/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:26.665426 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af38f3-857a-447e-91a1-9138fda0e1f7 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:26.674362 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068b741-6ad2-4b22-a807-9932e1ccffbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:26.674980 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068b741-6ad2-4b22-a807-9932e1ccffbb admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:26.675326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068b741-6ad2-4b22-a807-9932e1ccffbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:26.675767 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068b741-6ad2-4b22-a807-9932e1ccffbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:26.676633 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e068b741-6ad2-4b22-a807-9932e1ccffbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:26.678561 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e068b741-6ad2-4b22-a807-9932e1ccffbb admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:26.683917 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc5be7-7faf-4415-b9bf-3fd99043c6dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=967192ce42d64f9b886a6e5d7b194283, user_id=677da7e0e2ed4e72a66f0ed608177bc2, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:26.686738 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc5be7-7faf-4415-b9bf-3fd99043c6dc admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:26.691650 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1423/2868] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:41:26.696066 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068b741-6ad2-4b22-a807-9932e1ccffbb admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:26.701300 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1446/2869] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:41:26 2022] PUT /identity/v3/projects/967192ce42d64f9b886a6e5d7b194283/users/677da7e0e2ed4e72a66f0ed608177bc2/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:41:26.705062 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc5be7-7faf-4415-b9bf-3fd99043c6dc admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:26.707638 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068b741-6ad2-4b22-a807-9932e1ccffbb admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:26.767072 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076d8af-00b1-4e60-abf4-477bbe240015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:26.767735 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076d8af-00b1-4e60-abf4-477bbe240015 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:26.768015 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076d8af-00b1-4e60-abf4-477bbe240015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:26.768293 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076d8af-00b1-4e60-abf4-477bbe240015 admin admin] PATH_INFO: `/v3/projects/31e80682e55d41c588dec4c1a62cbc8c/users/004c2c401f6d45ef9f665e9fe68dd180/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:26.771300 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc5be7-7faf-4415-b9bf-3fd99043c6dc admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:26.776015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440139b-f95e-4b13-ae5d-cb76fc608a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:26.777168 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440139b-f95e-4b13-ae5d-cb76fc608a7f admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:26.777441 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440139b-f95e-4b13-ae5d-cb76fc608a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:26.777706 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440139b-f95e-4b13-ae5d-cb76fc608a7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:26.778516 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d440139b-f95e-4b13-ae5d-cb76fc608a7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:26.780129 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d440139b-f95e-4b13-ae5d-cb76fc608a7f admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:26.783534 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076d8af-00b1-4e60-abf4-477bbe240015 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31e80682e55d41c588dec4c1a62cbc8c, user_id=004c2c401f6d45ef9f665e9fe68dd180, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:26.786197 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076d8af-00b1-4e60-abf4-477bbe240015 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:26.790038 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1447/2870] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:41:26.794951 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440139b-f95e-4b13-ae5d-cb76fc608a7f admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:26.801244 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1424/2871] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:41:26 2022] PUT /identity/v3/projects/31e80682e55d41c588dec4c1a62cbc8c/users/004c2c401f6d45ef9f665e9fe68dd180/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:41:26.806572 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440139b-f95e-4b13-ae5d-cb76fc608a7f admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:26.812257 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb4f49-97ed-4523-a895-43182da8d746 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:26.812730 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb4f49-97ed-4523-a895-43182da8d746 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:26.813017 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb4f49-97ed-4523-a895-43182da8d746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:26.847301 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-e4cb4f49-97ed-4523-a895-43182da8d746 None None] MFA Rules not processed for user `004c2c401f6d45ef9f665e9fe68dd180`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:41:26.848698 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-e4cb4f49-97ed-4523-a895-43182da8d746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:26.857940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:41:26.861613 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:41:26.874182 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b471581-b483-4d09-8fab-2a6c5ad10fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:26.874885 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b471581-b483-4d09-8fab-2a6c5ad10fcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:26.875173 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b471581-b483-4d09-8fab-2a6c5ad10fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:26.875449 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b471581-b483-4d09-8fab-2a6c5ad10fcc admin admin] PATH_INFO: `/v3/projects/967192ce42d64f9b886a6e5d7b194283/users/677da7e0e2ed4e72a66f0ed608177bc2/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:26.877444 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b471581-b483-4d09-8fab-2a6c5ad10fcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=967192ce42d64f9b886a6e5d7b194283, user_id=677da7e0e2ed4e72a66f0ed608177bc2, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:26.880740 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b471581-b483-4d09-8fab-2a6c5ad10fcc admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:26.892688 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1448/2872] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:41:26 2022] PUT /identity/v3/projects/967192ce42d64f9b886a6e5d7b194283/users/677da7e0e2ed4e72a66f0ed608177bc2/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:41:26.922103 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1425/2873] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:41:26 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 10:41:26.934186 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b471581-b483-4d09-8fab-2a6c5ad10fcc admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:26.936624 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-7b471581-b483-4d09-8fab-2a6c5ad10fcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:26.999485 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b471581-b483-4d09-8fab-2a6c5ad10fcc admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:27.003614 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b616c-948c-4dca-aa64-f2729c851b7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feddae5ba018426697854931a55104e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:27.004225 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b616c-948c-4dca-aa64-f2729c851b7c service cinder] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:27.004502 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b616c-948c-4dca-aa64-f2729c851b7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:27.004768 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b616c-948c-4dca-aa64-f2729c851b7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:27.080542 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6b616c-948c-4dca-aa64-f2729c851b7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:27.082252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6b616c-948c-4dca-aa64-f2729c851b7c service cinder] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:27.093246 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1449/2874] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:41:26 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:41:31.482468 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56eeb5-2aa6-498a-a684-1abae74530c4 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:31.483012 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56eeb5-2aa6-498a-a684-1abae74530c4 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:31.483264 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56eeb5-2aa6-498a-a684-1abae74530c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:31.518426 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-1a56eeb5-2aa6-498a-a684-1abae74530c4 None None] MFA Rules not processed for user `677da7e0e2ed4e72a66f0ed608177bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:41:31.519764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-1a56eeb5-2aa6-498a-a684-1abae74530c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:31.597079 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1426/2875] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:41:31 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:41:31.613803 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b616c-948c-4dca-aa64-f2729c851b7c service cinder] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:31.615562 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-5c6b616c-948c-4dca-aa64-f2729c851b7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:31.714880 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b616c-948c-4dca-aa64-f2729c851b7c service cinder] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:31.720470 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd141a-6bda-472b-8586-eb7ce65e691f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:31.721069 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd141a-6bda-472b-8586-eb7ce65e691f service nova] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:31.721346 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd141a-6bda-472b-8586-eb7ce65e691f service nova] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:31.721620 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd141a-6bda-472b-8586-eb7ce65e691f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:31.734643 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:41:31.783852 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd141a-6bda-472b-8586-eb7ce65e691f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:31.785486 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd141a-6bda-472b-8586-eb7ce65e691f service nova] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:31.796201 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1450/2876] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:41:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:41:32.793378 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1715c01-147e-445e-aef2-5ef7edf292a9 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:32.793908 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1715c01-147e-445e-aef2-5ef7edf292a9 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:32.793908 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1715c01-147e-445e-aef2-5ef7edf292a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:32.812863 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-e1715c01-147e-445e-aef2-5ef7edf292a9 None None] MFA Rules not processed for user `c1dcec4c0b364d7caa37af361da31b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:41:32.814121 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-e1715c01-147e-445e-aef2-5ef7edf292a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:32.821047 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1427/2877] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:41:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:41:34.366067 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd141a-6bda-472b-8586-eb7ce65e691f service nova] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:34.436387 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd141a-6bda-472b-8586-eb7ce65e691f service nova] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:34.441404 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0decc503-a35b-4066-8ead-bb1af3cd4e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:34.442043 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0decc503-a35b-4066-8ead-bb1af3cd4e80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:34.442397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0decc503-a35b-4066-8ead-bb1af3cd4e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:34.442673 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0decc503-a35b-4066-8ead-bb1af3cd4e80 admin admin] PATH_INFO: `/v3/users/004c2c401f6d45ef9f665e9fe68dd180` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:34.443862 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0decc503-a35b-4066-8ead-bb1af3cd4e80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=004c2c401f6d45ef9f665e9fe68dd180)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:34.445780 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0decc503-a35b-4066-8ead-bb1af3cd4e80 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:34.499263 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-0decc503-a35b-4066-8ead-bb1af3cd4e80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '004c2c401f6d45ef9f665e9fe68dd180'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:34.499940 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-0decc503-a35b-4066-8ead-bb1af3cd4e80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '004c2c401f6d45ef9f665e9fe68dd180'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:34.507973 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-0decc503-a35b-4066-8ead-bb1af3cd4e80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '004c2c401f6d45ef9f665e9fe68dd180'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:34.520079 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-0decc503-a35b-4066-8ead-bb1af3cd4e80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '004c2c401f6d45ef9f665e9fe68dd180'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:34.520810 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-0decc503-a35b-4066-8ead-bb1af3cd4e80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '004c2c401f6d45ef9f665e9fe68dd180'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:34.535858 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1451/2878] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:41:34 2022] DELETE /identity/v3/users/004c2c401f6d45ef9f665e9fe68dd180 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:41:34.541369 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe15f8-e495-4087-807e-6dc3bb5f8d97 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:34.542547 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe15f8-e495-4087-807e-6dc3bb5f8d97 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:34.542547 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe15f8-e495-4087-807e-6dc3bb5f8d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:34.561854 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-3fbe15f8-e495-4087-807e-6dc3bb5f8d97 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:41:34.563122 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-3fbe15f8-e495-4087-807e-6dc3bb5f8d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:34.633769 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1428/2879] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:41:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:41:34.644167 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0decc503-a35b-4066-8ead-bb1af3cd4e80 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:34.645820 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-0decc503-a35b-4066-8ead-bb1af3cd4e80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:34.693619 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:41:34.717503 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0decc503-a35b-4066-8ead-bb1af3cd4e80 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:34.721156 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1277e-1634-482b-8e4a-88d49f1cc1bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:34.721760 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1277e-1634-482b-8e4a-88d49f1cc1bc service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:34.722068 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1277e-1634-482b-8e4a-88d49f1cc1bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:34.722347 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1277e-1634-482b-8e4a-88d49f1cc1bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:34.733231 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc1277e-1634-482b-8e4a-88d49f1cc1bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:34.734700 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc1277e-1634-482b-8e4a-88d49f1cc1bc service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:34.746540 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1452/2880] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:41:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:41:35.078770 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe15f8-e495-4087-807e-6dc3bb5f8d97 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:35.081065 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-3fbe15f8-e495-4087-807e-6dc3bb5f8d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:35.102954 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe15f8-e495-4087-807e-6dc3bb5f8d97 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:35.117844 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d5c86-7986-40ec-a1f7-1918d97b5c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:35.118533 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d5c86-7986-40ec-a1f7-1918d97b5c0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:35.118814 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d5c86-7986-40ec-a1f7-1918d97b5c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:35.119087 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d5c86-7986-40ec-a1f7-1918d97b5c0f admin admin] PATH_INFO: `/v3/projects/31e80682e55d41c588dec4c1a62cbc8c` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:35.120360 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d5c86-7986-40ec-a1f7-1918d97b5c0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=31e80682e55d41c588dec4c1a62cbc8c)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:35.122630 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d5c86-7986-40ec-a1f7-1918d97b5c0f admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:35.159956 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:41:35.193593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-680d5c86-7986-40ec-a1f7-1918d97b5c0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31e80682e55d41c588dec4c1a62cbc8c', 'request_id': 'req-680d5c86-7986-40ec-a1f7-1918d97b5c0f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:35.200144 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-680d5c86-7986-40ec-a1f7-1918d97b5c0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31e80682e55d41c588dec4c1a62cbc8c', 'request_id': 'req-680d5c86-7986-40ec-a1f7-1918d97b5c0f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:35.205845 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-680d5c86-7986-40ec-a1f7-1918d97b5c0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31e80682e55d41c588dec4c1a62cbc8c', 'request_id': 'req-680d5c86-7986-40ec-a1f7-1918d97b5c0f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:35.220409 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-680d5c86-7986-40ec-a1f7-1918d97b5c0f admin admin] The token cache is being invalidate because project 31e80682e55d41c588dec4c1a62cbc8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:41:35.220721 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-680d5c86-7986-40ec-a1f7-1918d97b5c0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:35.221340 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-680d5c86-7986-40ec-a1f7-1918d97b5c0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:41:35.223004 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1429/2881] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:41:35 2022] DELETE /identity/v3/projects/31e80682e55d41c588dec4c1a62cbc8c => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 10:41:35.230898 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d74a1-5af6-458b-8787-ff755f9d0b40 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:35.231265 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d74a1-5af6-458b-8787-ff755f9d0b40 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:35.231611 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d74a1-5af6-458b-8787-ff755f9d0b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:35.256150 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-212d74a1-5af6-458b-8787-ff755f9d0b40 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:41:35.257421 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-212d74a1-5af6-458b-8787-ff755f9d0b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:35.320951 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1453/2882] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:41:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:41:35.329045 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01017c7c-3fd7-4c04-b4d0-78ced4bf2c76 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:35.329326 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01017c7c-3fd7-4c04-b4d0-78ced4bf2c76 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:35.329598 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01017c7c-3fd7-4c04-b4d0-78ced4bf2c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:35.352009 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-01017c7c-3fd7-4c04-b4d0-78ced4bf2c76 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:41:35.353703 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-01017c7c-3fd7-4c04-b4d0-78ced4bf2c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:35.364322 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1430/2883] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:41:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:41:35.368881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d74a1-5af6-458b-8787-ff755f9d0b40 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:35.381039 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d74a1-5af6-458b-8787-ff755f9d0b40 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:35.393794 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4527b753-c05e-468c-8dda-4ec813edcceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:35.394369 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527b753-c05e-468c-8dda-4ec813edcceb admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:35.394644 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527b753-c05e-468c-8dda-4ec813edcceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:35.394911 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527b753-c05e-468c-8dda-4ec813edcceb admin admin] PATH_INFO: `/v3/domains` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:35.395580 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4527b753-c05e-468c-8dda-4ec813edcceb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81420) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 10:41:35.395871 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4527b753-c05e-468c-8dda-4ec813edcceb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:35.397464 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4527b753-c05e-468c-8dda-4ec813edcceb admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:35.412056 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1454/2884] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Sep 26 10:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 10:41:35.415342 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01017c7c-3fd7-4c04-b4d0-78ced4bf2c76 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:35.426237 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01017c7c-3fd7-4c04-b4d0-78ced4bf2c76 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:35.431139 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd016d-4e75-4f38-a6df-87c82bbfbe6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:35.431745 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd016d-4e75-4f38-a6df-87c82bbfbe6e admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:35.432025 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd016d-4e75-4f38-a6df-87c82bbfbe6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:35.432305 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd016d-4e75-4f38-a6df-87c82bbfbe6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:35.432972 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abd016d-4e75-4f38-a6df-87c82bbfbe6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:35.435049 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abd016d-4e75-4f38-a6df-87c82bbfbe6e admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:35.471003 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1431/2885] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:41:35 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:41:35.475303 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4527b753-c05e-468c-8dda-4ec813edcceb admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:35.486250 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4527b753-c05e-468c-8dda-4ec813edcceb admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:35.551782 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7377b9b2-7340-4b5c-b08f-e0cd5e420f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:35.552432 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7377b9b2-7340-4b5c-b08f-e0cd5e420f99 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:35.552709 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7377b9b2-7340-4b5c-b08f-e0cd5e420f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:35.552980 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7377b9b2-7340-4b5c-b08f-e0cd5e420f99 admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:35.553627 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7377b9b2-7340-4b5c-b08f-e0cd5e420f99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:35.555715 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7377b9b2-7340-4b5c-b08f-e0cd5e420f99 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:35.582065 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1455/2886] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:41:35 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:41:35.583255 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd016d-4e75-4f38-a6df-87c82bbfbe6e admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:35.595602 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd016d-4e75-4f38-a6df-87c82bbfbe6e admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:35.600945 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b762e-f18a-4455-af75-7cfabe8eee75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:35.601662 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b762e-f18a-4455-af75-7cfabe8eee75 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:35.601946 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b762e-f18a-4455-af75-7cfabe8eee75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:35.602520 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b762e-f18a-4455-af75-7cfabe8eee75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:35.603179 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55b762e-f18a-4455-af75-7cfabe8eee75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:35.605364 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55b762e-f18a-4455-af75-7cfabe8eee75 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:35.616677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1432/2887] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:41:35.621595 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7377b9b2-7340-4b5c-b08f-e0cd5e420f99 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:35.632332 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7377b9b2-7340-4b5c-b08f-e0cd5e420f99 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:35.637229 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6600e7dd-10d5-4948-b3c9-d38b12a6ed2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:35.637839 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600e7dd-10d5-4948-b3c9-d38b12a6ed2c admin admin] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:35.638159 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600e7dd-10d5-4948-b3c9-d38b12a6ed2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:35.638434 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600e7dd-10d5-4948-b3c9-d38b12a6ed2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:35.639078 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6600e7dd-10d5-4948-b3c9-d38b12a6ed2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:35.640667 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6600e7dd-10d5-4948-b3c9-d38b12a6ed2c admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:35.652226 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1456/2888] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:41:35.656405 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b762e-f18a-4455-af75-7cfabe8eee75 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:35.668077 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b762e-f18a-4455-af75-7cfabe8eee75 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:35.672917 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e356c5-0b46-45ad-91f4-499947befe94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:35.673597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e356c5-0b46-45ad-91f4-499947befe94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:35.673874 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e356c5-0b46-45ad-91f4-499947befe94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:35.674359 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e356c5-0b46-45ad-91f4-499947befe94 admin admin] PATH_INFO: `/v3/projects/1e0f8c6af4d3456c9d700088cd8f5eb5/users/0561a33fd6984118b1fd221f9eaaeef6/roles/dc7cfc688d354df4bc870fe8621c48bd` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:35.691836 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e356c5-0b46-45ad-91f4-499947befe94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e0f8c6af4d3456c9d700088cd8f5eb5, user_id=0561a33fd6984118b1fd221f9eaaeef6, role_id=dc7cfc688d354df4bc870fe8621c48bd)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:35.694525 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e356c5-0b46-45ad-91f4-499947befe94 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:35.707009 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1433/2889] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:41:35 2022] PUT /identity/v3/projects/1e0f8c6af4d3456c9d700088cd8f5eb5/users/0561a33fd6984118b1fd221f9eaaeef6/roles/dc7cfc688d354df4bc870fe8621c48bd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:41:35.712179 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f0901-0022-4736-b171-f36c671fe075 None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:35.712467 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f0901-0022-4736-b171-f36c671fe075 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:35.712745 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f0901-0022-4736-b171-f36c671fe075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:35.732129 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-609f0901-0022-4736-b171-f36c671fe075 None None] MFA Rules not processed for user `7ecff1fbd79248b7bcd1e69035be3515`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:41:35.733344 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-609f0901-0022-4736-b171-f36c671fe075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:35.776126 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:41:35.796620 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1457/2890] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:41:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:41:35.806478 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e356c5-0b46-45ad-91f4-499947befe94 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:35.809007 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-25e356c5-0b46-45ad-91f4-499947befe94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:35.879756 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e356c5-0b46-45ad-91f4-499947befe94 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:35.883753 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e66b6-a584-421a-946a-8595f9a1fbcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:35.884448 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e66b6-a584-421a-946a-8595f9a1fbcf service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:35.884796 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e66b6-a584-421a-946a-8595f9a1fbcf service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:35.885215 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e66b6-a584-421a-946a-8595f9a1fbcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:35.900147 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e66b6-a584-421a-946a-8595f9a1fbcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:35.901611 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e66b6-a584-421a-946a-8595f9a1fbcf service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:35.912598 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1434/2891] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:41:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:41:40.167520 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a3a0c-8899-43a2-ae1b-eb860621ffad None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:40.168003 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a3a0c-8899-43a2-ae1b-eb860621ffad None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:40.168092 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a3a0c-8899-43a2-ae1b-eb860621ffad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:40.199482 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-376a3a0c-8899-43a2-ae1b-eb860621ffad None None] MFA Rules not processed for user `0561a33fd6984118b1fd221f9eaaeef6`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:41:40.200722 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-376a3a0c-8899-43a2-ae1b-eb860621ffad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:40.268047 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1458/2892] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:41:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:41:40.272618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e66b6-a584-421a-946a-8595f9a1fbcf service neutron] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:40.283799 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e66b6-a584-421a-946a-8595f9a1fbcf service neutron] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:40.289587 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d89f26-e7e5-4c96-acf0-2c1f699c0c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:40.290441 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d89f26-e7e5-4c96-acf0-2c1f699c0c63 admin admin] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:40.290793 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d89f26-e7e5-4c96-acf0-2c1f699c0c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:40.291136 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d89f26-e7e5-4c96-acf0-2c1f699c0c63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:40.291892 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d89f26-e7e5-4c96-acf0-2c1f699c0c63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:40.293879 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d89f26-e7e5-4c96-acf0-2c1f699c0c63 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:40.323024 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1435/2893] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Sep 26 10:41:40 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 10:41:40.327898 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a3a0c-8899-43a2-ae1b-eb860621ffad None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:40.340352 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a3a0c-8899-43a2-ae1b-eb860621ffad None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:40.403641 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312bf84-d27c-44f7-946a-44ab27b68cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:40.404253 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312bf84-d27c-44f7-946a-44ab27b68cc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:40.404535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312bf84-d27c-44f7-946a-44ab27b68cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:40.404809 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312bf84-d27c-44f7-946a-44ab27b68cc6 admin admin] PATH_INFO: `/v3/users` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:40.405454 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2312bf84-d27c-44f7-946a-44ab27b68cc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:40.407406 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2312bf84-d27c-44f7-946a-44ab27b68cc6 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:40.436112 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1459/2894] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Sep 26 10:41:40 2022] POST /identity/v3/users => generated 295 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 10:41:40.437792 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d89f26-e7e5-4c96-acf0-2c1f699c0c63 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:40.449716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d89f26-e7e5-4c96-acf0-2c1f699c0c63 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:40.455600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37c8b5-cd22-4eaa-a1ce-5be0ef10b98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:40.456341 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37c8b5-cd22-4eaa-a1ce-5be0ef10b98a admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:40.456693 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37c8b5-cd22-4eaa-a1ce-5be0ef10b98a admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:40.457036 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37c8b5-cd22-4eaa-a1ce-5be0ef10b98a admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:40.457676 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c37c8b5-cd22-4eaa-a1ce-5be0ef10b98a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:40.459432 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c37c8b5-cd22-4eaa-a1ce-5be0ef10b98a admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:40.473694 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1436/2895] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:41:40.475202 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312bf84-d27c-44f7-946a-44ab27b68cc6 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:40.485429 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312bf84-d27c-44f7-946a-44ab27b68cc6 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:40.490702 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48566cd-a744-4a07-95ef-8b35c73b659d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:40.491131 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48566cd-a744-4a07-95ef-8b35c73b659d admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:40.491411 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48566cd-a744-4a07-95ef-8b35c73b659d admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:40.491686 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48566cd-a744-4a07-95ef-8b35c73b659d admin admin] PATH_INFO: `/v3/projects/e9f6ccbccd7a43b5bc1212cc59699e8d/users/d3a51714e44e45e9bd5d49a65660ac45/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:40.515125 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48566cd-a744-4a07-95ef-8b35c73b659d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9f6ccbccd7a43b5bc1212cc59699e8d, user_id=d3a51714e44e45e9bd5d49a65660ac45, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:40.517990 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48566cd-a744-4a07-95ef-8b35c73b659d admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:40.532663 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1460/2896] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:41:40 2022] PUT /identity/v3/projects/e9f6ccbccd7a43b5bc1212cc59699e8d/users/d3a51714e44e45e9bd5d49a65660ac45/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:41:40.536939 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37c8b5-cd22-4eaa-a1ce-5be0ef10b98a admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:40.547047 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37c8b5-cd22-4eaa-a1ce-5be0ef10b98a admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:40.559886 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:41:40.610754 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b9985-fc6e-4122-8732-2146fa326ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:40.611630 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b9985-fc6e-4122-8732-2146fa326ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:40.611906 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b9985-fc6e-4122-8732-2146fa326ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:40.612178 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b9985-fc6e-4122-8732-2146fa326ad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:40.612822 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3b9985-fc6e-4122-8732-2146fa326ad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:40.614774 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3b9985-fc6e-4122-8732-2146fa326ad2 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:40.626904 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1437/2897] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Sep 26 10:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 10:41:40.631130 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48566cd-a744-4a07-95ef-8b35c73b659d admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:40.641966 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48566cd-a744-4a07-95ef-8b35c73b659d admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:40.646650 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd0a1e-a24c-4eb1-adaf-d9e1ab435f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:40.647294 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd0a1e-a24c-4eb1-adaf-d9e1ab435f31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:40.647577 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd0a1e-a24c-4eb1-adaf-d9e1ab435f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:40.647851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd0a1e-a24c-4eb1-adaf-d9e1ab435f31 admin admin] PATH_INFO: `/v3/projects/e9f6ccbccd7a43b5bc1212cc59699e8d/users/d3a51714e44e45e9bd5d49a65660ac45/roles/532b37d9076c45618cf352aafb85f35e` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:40.649785 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd0a1e-a24c-4eb1-adaf-d9e1ab435f31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9f6ccbccd7a43b5bc1212cc59699e8d, user_id=d3a51714e44e45e9bd5d49a65660ac45, role_id=532b37d9076c45618cf352aafb85f35e)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:40.652476 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd0a1e-a24c-4eb1-adaf-d9e1ab435f31 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:40.663015 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1461/2898] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Sep 26 10:41:40 2022] PUT /identity/v3/projects/e9f6ccbccd7a43b5bc1212cc59699e8d/users/d3a51714e44e45e9bd5d49a65660ac45/roles/532b37d9076c45618cf352aafb85f35e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:41:44.850664 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb80714-af38-4a9c-b40a-8e8692975cad None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:44.851537 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb80714-af38-4a9c-b40a-8e8692975cad None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:44.851992 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb80714-af38-4a9c-b40a-8e8692975cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:44.902122 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-adb80714-af38-4a9c-b40a-8e8692975cad None None] MFA Rules not processed for user `d3a51714e44e45e9bd5d49a65660ac45`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:41:44.903371 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-adb80714-af38-4a9c-b40a-8e8692975cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:45.051687 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1438/2899] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:41:44 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:41:45.061774 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd0a1e-a24c-4eb1-adaf-d9e1ab435f31 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:41:45.064117 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-89fd0a1e-a24c-4eb1-adaf-d9e1ab435f31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:41:45.113863 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:41:45.199518 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd0a1e-a24c-4eb1-adaf-d9e1ab435f31 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:41:45.219142 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a105b44-0c60-4190-9bfc-77dc1e868ac4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:41:45.220008 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a105b44-0c60-4190-9bfc-77dc1e868ac4 service nova] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:41:45.220471 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a105b44-0c60-4190-9bfc-77dc1e868ac4 service nova] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:41:45.220898 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a105b44-0c60-4190-9bfc-77dc1e868ac4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:41:45.326669 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a105b44-0c60-4190-9bfc-77dc1e868ac4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:41:45.328528 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a105b44-0c60-4190-9bfc-77dc1e868ac4 service nova] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:41:45.339764 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1462/2900] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:41:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:41:55.037738 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=55 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:41:55.334487 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=44 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:42:02.851951 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fe3d9-06e4-41f5-ac83-1bb6a772070e None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:42:02.852463 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fe3d9-06e4-41f5-ac83-1bb6a772070e None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:42:02.852567 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fe3d9-06e4-41f5-ac83-1bb6a772070e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:42:02.873401 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-7b6fe3d9-06e4-41f5-ac83-1bb6a772070e None None] MFA Rules not processed for user `0561a33fd6984118b1fd221f9eaaeef6`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:42:02.874646 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-7b6fe3d9-06e4-41f5-ac83-1bb6a772070e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:42:02.881614 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1439/2901] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:42:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:42:05.971151 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a105b44-0c60-4190-9bfc-77dc1e868ac4 service nova] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:42:05.972811 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-6a105b44-0c60-4190-9bfc-77dc1e868ac4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:42:06.040182 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a105b44-0c60-4190-9bfc-77dc1e868ac4 service nova] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:42:06.043865 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55623cbe-04ba-4615-ba03-5186b520f723 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:42:06.044469 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55623cbe-04ba-4615-ba03-5186b520f723 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:42:06.044747 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55623cbe-04ba-4615-ba03-5186b520f723 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:42:06.045026 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55623cbe-04ba-4615-ba03-5186b520f723 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:42:06.046876 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-55623cbe-04ba-4615-ba03-5186b520f723 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:42:06.114172 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55623cbe-04ba-4615-ba03-5186b520f723 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:42:06.115869 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55623cbe-04ba-4615-ba03-5186b520f723 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:42:06.125620 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1463/2902] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:42:05 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:42:12.867388 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=3 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:42:16.121112 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=50 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:42:19.532352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6fe3d9-06e4-41f5-ac83-1bb6a772070e None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:42:19.542005 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6fe3d9-06e4-41f5-ac83-1bb6a772070e None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:42:19.546125 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c5223-866a-4dbe-b908-12ef411cfca3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:42:19.546716 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c5223-866a-4dbe-b908-12ef411cfca3 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:42:19.547011 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c5223-866a-4dbe-b908-12ef411cfca3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:42:19.547264 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c5223-866a-4dbe-b908-12ef411cfca3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:42:19.556031 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3c5223-866a-4dbe-b908-12ef411cfca3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:42:19.557442 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3c5223-866a-4dbe-b908-12ef411cfca3 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:42:19.566781 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1440/2903] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:42:19 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:42:29.562694 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=6 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:43:49.106445 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55623cbe-04ba-4615-ba03-5186b520f723 service neutron] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:43:49.116054 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55623cbe-04ba-4615-ba03-5186b520f723 service neutron] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:43:49.119741 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c358a2-31fd-42be-ad00-137416040d19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:43:49.120331 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c358a2-31fd-42be-ad00-137416040d19 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:43:49.120612 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c358a2-31fd-42be-ad00-137416040d19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:43:49.120888 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c358a2-31fd-42be-ad00-137416040d19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:43:49.122702 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-e2c358a2-31fd-42be-ad00-137416040d19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:43:49.132162 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c358a2-31fd-42be-ad00-137416040d19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:43:49.133587 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c358a2-31fd-42be-ad00-137416040d19 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:43:49.141614 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1464/2904] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:43:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:43:59.138062 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=6 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:44:05.659329 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c5223-866a-4dbe-b908-12ef411cfca3 service neutron] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:44:05.668749 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c5223-866a-4dbe-b908-12ef411cfca3 service neutron] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:44:05.672406 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a48bb-d753-4dd4-9e20-45daea4fd001 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:44:05.672989 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a48bb-d753-4dd4-9e20-45daea4fd001 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:44:05.673263 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a48bb-d753-4dd4-9e20-45daea4fd001 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:44:05.673533 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a48bb-d753-4dd4-9e20-45daea4fd001 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:44:05.675329 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-195a48bb-d753-4dd4-9e20-45daea4fd001 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:44:05.689759 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a48bb-d753-4dd4-9e20-45daea4fd001 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:44:05.691248 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a48bb-d753-4dd4-9e20-45daea4fd001 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:44:05.702332 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1441/2905] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:44:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:44:06.884552 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c358a2-31fd-42be-ad00-137416040d19 service neutron] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:44:06.893276 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c358a2-31fd-42be-ad00-137416040d19 service neutron] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:44:06.896948 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96525e58-c955-449f-9e8c-792e782d4d18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:44:06.897530 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96525e58-c955-449f-9e8c-792e782d4d18 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:44:06.897804 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96525e58-c955-449f-9e8c-792e782d4d18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:44:06.898104 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96525e58-c955-449f-9e8c-792e782d4d18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:44:06.900866 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-96525e58-c955-449f-9e8c-792e782d4d18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:44:06.914477 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96525e58-c955-449f-9e8c-792e782d4d18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:44:06.915889 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96525e58-c955-449f-9e8c-792e782d4d18 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:44:06.928528 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1465/2906] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:44:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:44:15.697423 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=6 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:44:16.923040 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=6 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:44:20.205980 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14c0ef-7eaf-4ad6-b274-ea37428cef0d None None] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:44:20.206559 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14c0ef-7eaf-4ad6-b274-ea37428cef0d None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:44:20.206559 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14c0ef-7eaf-4ad6-b274-ea37428cef0d None None] PATH_INFO: `/` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:44:20.208426 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1442/2907] 10.222.0.49 () {54 vars in 921 bytes} [Mon Sep 26 10:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:44:20.214211 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7809866-dff7-4d86-8c1b-cadd5a9635fb None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:44:20.214683 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7809866-dff7-4d86-8c1b-cadd5a9635fb None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:44:20.214794 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7809866-dff7-4d86-8c1b-cadd5a9635fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:44:20.254365 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-b7809866-dff7-4d86-8c1b-cadd5a9635fb None None] MFA Rules not processed for user `f841bf31c43b4333a3d834d38b5197e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:44:20.256174 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-b7809866-dff7-4d86-8c1b-cadd5a9635fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:44:20.263039 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1466/2908] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Sep 26 10:44:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:44:20.278553 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14c0ef-7eaf-4ad6-b274-ea37428cef0d None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:44:20.290593 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14c0ef-7eaf-4ad6-b274-ea37428cef0d None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:44:20.294153 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3d4b5-9e03-462f-963f-fb11729cd308 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:44:20.294748 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3d4b5-9e03-462f-963f-fb11729cd308 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:44:20.295027 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3d4b5-9e03-462f-963f-fb11729cd308 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:44:20.295297 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3d4b5-9e03-462f-963f-fb11729cd308 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:44:20.302662 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3d4b5-9e03-462f-963f-fb11729cd308 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:44:20.304081 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3d4b5-9e03-462f-963f-fb11729cd308 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:44:20.312128 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1443/2909] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:44:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:44:20.403691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961b424-d02d-48fc-879a-ec243bf99d70 None None] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:44:20.403915 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961b424-d02d-48fc-879a-ec243bf99d70 None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:44:20.404163 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961b424-d02d-48fc-879a-ec243bf99d70 None None] PATH_INFO: `/` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:44:20.406337 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1467/2910] 10.222.0.49 () {54 vars in 921 bytes} [Mon Sep 26 10:44:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 10:44:20.412780 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20723d-9734-4063-b042-d304abaaa448 None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:44:20.413086 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20723d-9734-4063-b042-d304abaaa448 None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:44:20.413347 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20723d-9734-4063-b042-d304abaaa448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:44:20.455620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-3b20723d-9734-4063-b042-d304abaaa448 None None] MFA Rules not processed for user `cee41014d5d043888c6b399710bccab4`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:44:20.456861 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-3b20723d-9734-4063-b042-d304abaaa448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:44:20.466783 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1444/2911] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Sep 26 10:44:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:44:20.477864 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961b424-d02d-48fc-879a-ec243bf99d70 None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:44:20.490118 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961b424-d02d-48fc-879a-ec243bf99d70 None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:44:20.493611 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a09d0-f77b-478e-9af9-7868f9c7766f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:44:20.494266 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a09d0-f77b-478e-9af9-7868f9c7766f service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:44:20.494547 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a09d0-f77b-478e-9af9-7868f9c7766f service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:44:20.494816 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a09d0-f77b-478e-9af9-7868f9c7766f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:44:20.505208 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a09d0-f77b-478e-9af9-7868f9c7766f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:44:20.506615 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a09d0-f77b-478e-9af9-7868f9c7766f service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:44:20.517565 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1468/2912] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:44:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:44:20.579625 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20723d-9734-4063-b042-d304abaaa448 None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:44:20.581187 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-3b20723d-9734-4063-b042-d304abaaa448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:44:20.654656 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20723d-9734-4063-b042-d304abaaa448 None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:44:20.659741 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e652c-58db-4b35-be6d-3ed2e0607401 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84b02e7351914cabb67462454b328d05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:44:20.660339 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e652c-58db-4b35-be6d-3ed2e0607401 service placement] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:44:20.660620 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e652c-58db-4b35-be6d-3ed2e0607401 service placement] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:44:20.661414 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e652c-58db-4b35-be6d-3ed2e0607401 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:44:20.663208 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-8d0e652c-58db-4b35-be6d-3ed2e0607401 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:44:20.674907 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e652c-58db-4b35-be6d-3ed2e0607401 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:44:20.676516 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e652c-58db-4b35-be6d-3ed2e0607401 service placement] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:44:20.687370 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1445/2913] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:44:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:44:21.303043 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a09d0-f77b-478e-9af9-7868f9c7766f service neutron] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:44:21.314872 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a09d0-f77b-478e-9af9-7868f9c7766f service neutron] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:44:21.321966 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb4391d-3b1c-4292-8e10-c04229d6997a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:44:21.322680 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4391d-3b1c-4292-8e10-c04229d6997a service nova] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:44:21.323018 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4391d-3b1c-4292-8e10-c04229d6997a service nova] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:44:21.323293 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4391d-3b1c-4292-8e10-c04229d6997a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:44:21.325168 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-7bb4391d-3b1c-4292-8e10-c04229d6997a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:44:21.340527 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb4391d-3b1c-4292-8e10-c04229d6997a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:44:21.342060 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb4391d-3b1c-4292-8e10-c04229d6997a service nova] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:44:21.355084 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1469/2914] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:44:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:44:23.231264 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a911d-613f-4dcb-a62b-a5f50cebd08b None None] REQUEST_METHOD: `POST` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:44:23.231264 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a911d-613f-4dcb-a62b-a5f50cebd08b None None] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:44:23.231264 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a911d-613f-4dcb-a62b-a5f50cebd08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:44:23.251568 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.auth.core [None req-a54a911d-613f-4dcb-a62b-a5f50cebd08b None None] MFA Rules not processed for user `0561a33fd6984118b1fd221f9eaaeef6`. Rule list: `[]` (Enabled: `True`). {{(pid=81421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:44:23.252783 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-a54a911d-613f-4dcb-a62b-a5f50cebd08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:44:23.262999 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1446/2915] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:44:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:44:23.576502 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb4391d-3b1c-4292-8e10-c04229d6997a service nova] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:44:23.586567 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb4391d-3b1c-4292-8e10-c04229d6997a service nova] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:44:23.590105 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d11ea-491f-4c5d-b1ad-af01b592b639 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:44:23.590715 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d11ea-491f-4c5d-b1ad-af01b592b639 service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:44:23.590976 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d11ea-491f-4c5d-b1ad-af01b592b639 service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:44:23.591250 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d11ea-491f-4c5d-b1ad-af01b592b639 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:44:23.592978 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-c31d11ea-491f-4c5d-b1ad-af01b592b639 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:44:23.603521 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d11ea-491f-4c5d-b1ad-af01b592b639 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:44:23.604935 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d11ea-491f-4c5d-b1ad-af01b592b639 service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:44:23.613461 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1470/2916] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:44:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:44:23.680216 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a911d-613f-4dcb-a62b-a5f50cebd08b None None] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:44:23.694946 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a911d-613f-4dcb-a62b-a5f50cebd08b None None] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:44:23.702362 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706178e6-6585-4730-8464-004785c9c77d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:44:23.702974 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706178e6-6585-4730-8464-004785c9c77d service nova] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:44:23.703249 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706178e6-6585-4730-8464-004785c9c77d service nova] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:44:23.703694 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706178e6-6585-4730-8464-004785c9c77d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:44:23.706058 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-706178e6-6585-4730-8464-004785c9c77d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:44:23.720103 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706178e6-6585-4730-8464-004785c9c77d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:44:23.722099 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706178e6-6585-4730-8464-004785c9c77d service nova] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:44:23.733403 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1447/2917] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:44:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:44:33.609382 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=27 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:44:33.728535 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=56 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:44:41.594080 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d11ea-491f-4c5d-b1ad-af01b592b639 service neutron] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:44:41.607535 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d11ea-491f-4c5d-b1ad-af01b592b639 service neutron] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:44:41.613757 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755cab65-73d3-401c-8410-16f0085a1cef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee41014d5d043888c6b399710bccab4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:44:41.614388 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755cab65-73d3-401c-8410-16f0085a1cef service nova] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:44:41.614665 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755cab65-73d3-401c-8410-16f0085a1cef service nova] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:44:41.614938 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755cab65-73d3-401c-8410-16f0085a1cef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:44:41.616664 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-755cab65-73d3-401c-8410-16f0085a1cef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:44:41.629580 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755cab65-73d3-401c-8410-16f0085a1cef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:44:41.631488 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755cab65-73d3-401c-8410-16f0085a1cef service nova] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:44:41.643224 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1471/2918] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:44:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:44:51.638183 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=6 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:45:50.820941 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706178e6-6585-4730-8464-004785c9c77d service nova] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:45:50.822376 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-706178e6-6585-4730-8464-004785c9c77d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:45:50.836367 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706178e6-6585-4730-8464-004785c9c77d service nova] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:45:50.841302 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed5abf-2fca-4d8b-8990-d8e3713f2d08 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84b02e7351914cabb67462454b328d05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:45:50.841885 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed5abf-2fca-4d8b-8990-d8e3713f2d08 service placement] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:45:50.842199 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed5abf-2fca-4d8b-8990-d8e3713f2d08 service placement] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:45:50.842469 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed5abf-2fca-4d8b-8990-d8e3713f2d08 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:45:50.844139 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-27ed5abf-2fca-4d8b-8990-d8e3713f2d08 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:45:50.855880 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed5abf-2fca-4d8b-8990-d8e3713f2d08 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:45:50.857387 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed5abf-2fca-4d8b-8990-d8e3713f2d08 service placement] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:45:50.868568 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1448/2919] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:45:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:45:57.757571 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755cab65-73d3-401c-8410-16f0085a1cef service nova] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:45:57.759083 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-755cab65-73d3-401c-8410-16f0085a1cef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:45:57.833669 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755cab65-73d3-401c-8410-16f0085a1cef service nova] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:45:57.838188 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ce3ba-ca1a-40ee-ab07-87aff59be36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:45:57.838827 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ce3ba-ca1a-40ee-ab07-87aff59be36a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:45:57.839107 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ce3ba-ca1a-40ee-ab07-87aff59be36a admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:45:57.839385 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ce3ba-ca1a-40ee-ab07-87aff59be36a admin admin] PATH_INFO: `/v3/users/c1dcec4c0b364d7caa37af361da31b18` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:45:57.840519 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ce3ba-ca1a-40ee-ab07-87aff59be36a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1dcec4c0b364d7caa37af361da31b18)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:45:57.842475 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ce3ba-ca1a-40ee-ab07-87aff59be36a admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:45:57.896349 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f06ce3ba-ca1a-40ee-ab07-87aff59be36a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1dcec4c0b364d7caa37af361da31b18'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:45:57.896999 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f06ce3ba-ca1a-40ee-ab07-87aff59be36a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1dcec4c0b364d7caa37af361da31b18'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:45:57.903926 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f06ce3ba-ca1a-40ee-ab07-87aff59be36a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1dcec4c0b364d7caa37af361da31b18'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:45:57.916551 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f06ce3ba-ca1a-40ee-ab07-87aff59be36a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1dcec4c0b364d7caa37af361da31b18'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:45:57.917463 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-f06ce3ba-ca1a-40ee-ab07-87aff59be36a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1dcec4c0b364d7caa37af361da31b18'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:45:57.932171 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1472/2920] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:45:57 2022] DELETE /identity/v3/users/c1dcec4c0b364d7caa37af361da31b18 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:45:58.132389 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed5abf-2fca-4d8b-8990-d8e3713f2d08 service placement] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:45:58.145397 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed5abf-2fca-4d8b-8990-d8e3713f2d08 service placement] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:45:58.205124 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9337ca-3476-414d-8f9b-87f934d9c470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:45:58.205744 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9337ca-3476-414d-8f9b-87f934d9c470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:45:58.206043 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9337ca-3476-414d-8f9b-87f934d9c470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:45:58.206322 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9337ca-3476-414d-8f9b-87f934d9c470 admin admin] PATH_INFO: `/v3/projects/b35e6a97e32b426daab0ba663323e8a8` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:45:58.207476 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9337ca-3476-414d-8f9b-87f934d9c470 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b35e6a97e32b426daab0ba663323e8a8)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:45:58.209352 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9337ca-3476-414d-8f9b-87f934d9c470 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:45:58.255117 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-0e9337ca-3476-414d-8f9b-87f934d9c470 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b35e6a97e32b426daab0ba663323e8a8', 'request_id': 'req-0e9337ca-3476-414d-8f9b-87f934d9c470'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:45:58.261102 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-0e9337ca-3476-414d-8f9b-87f934d9c470 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b35e6a97e32b426daab0ba663323e8a8', 'request_id': 'req-0e9337ca-3476-414d-8f9b-87f934d9c470'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:45:58.266689 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-0e9337ca-3476-414d-8f9b-87f934d9c470 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b35e6a97e32b426daab0ba663323e8a8', 'request_id': 'req-0e9337ca-3476-414d-8f9b-87f934d9c470'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:45:58.278389 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-0e9337ca-3476-414d-8f9b-87f934d9c470 admin admin] The token cache is being invalidate because project b35e6a97e32b426daab0ba663323e8a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:45:58.278677 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-0e9337ca-3476-414d-8f9b-87f934d9c470 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:45:58.279300 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-0e9337ca-3476-414d-8f9b-87f934d9c470 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:45:58.280764 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1449/2921] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:45:58 2022] DELETE /identity/v3/projects/b35e6a97e32b426daab0ba663323e8a8 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:45:58.284823 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ce3ba-ca1a-40ee-ab07-87aff59be36a admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:45:58.286248 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-f06ce3ba-ca1a-40ee-ab07-87aff59be36a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:45:58.299581 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ce3ba-ca1a-40ee-ab07-87aff59be36a admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:45:58.355564 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8f59e-e87a-42fe-921c-055eaffc012d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:45:58.356182 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8f59e-e87a-42fe-921c-055eaffc012d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:45:58.356460 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8f59e-e87a-42fe-921c-055eaffc012d admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:45:58.356733 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8f59e-e87a-42fe-921c-055eaffc012d admin admin] PATH_INFO: `/v3/users/677da7e0e2ed4e72a66f0ed608177bc2` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:45:58.357851 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef8f59e-e87a-42fe-921c-055eaffc012d admin admin] RBAC: Authorizing `identity:delete_user(user_id=677da7e0e2ed4e72a66f0ed608177bc2)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:45:58.360533 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef8f59e-e87a-42fe-921c-055eaffc012d admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:45:58.382206 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:45:58.406706 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-3ef8f59e-e87a-42fe-921c-055eaffc012d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '677da7e0e2ed4e72a66f0ed608177bc2'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:45:58.407344 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-3ef8f59e-e87a-42fe-921c-055eaffc012d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '677da7e0e2ed4e72a66f0ed608177bc2'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:45:58.414282 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-3ef8f59e-e87a-42fe-921c-055eaffc012d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '677da7e0e2ed4e72a66f0ed608177bc2'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:45:58.426545 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-3ef8f59e-e87a-42fe-921c-055eaffc012d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '677da7e0e2ed4e72a66f0ed608177bc2'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:45:58.427173 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-3ef8f59e-e87a-42fe-921c-055eaffc012d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '677da7e0e2ed4e72a66f0ed608177bc2'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:45:58.439852 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1473/2922] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:45:58 2022] DELETE /identity/v3/users/677da7e0e2ed4e72a66f0ed608177bc2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:45:58.615660 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9337ca-3476-414d-8f9b-87f934d9c470 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:45:58.617222 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-0e9337ca-3476-414d-8f9b-87f934d9c470 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:45:58.633465 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9337ca-3476-414d-8f9b-87f934d9c470 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:45:58.690455 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8744d-f33f-4d05-a9ef-e722f574cd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:45:58.691077 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8744d-f33f-4d05-a9ef-e722f574cd7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:45:58.691354 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8744d-f33f-4d05-a9ef-e722f574cd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:45:58.691621 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8744d-f33f-4d05-a9ef-e722f574cd7f admin admin] PATH_INFO: `/v3/projects/967192ce42d64f9b886a6e5d7b194283` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:45:58.692814 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a8744d-f33f-4d05-a9ef-e722f574cd7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=967192ce42d64f9b886a6e5d7b194283)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:45:58.694785 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a8744d-f33f-4d05-a9ef-e722f574cd7f admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:45:58.715801 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:45:58.742594 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-83a8744d-f33f-4d05-a9ef-e722f574cd7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '967192ce42d64f9b886a6e5d7b194283', 'request_id': 'req-83a8744d-f33f-4d05-a9ef-e722f574cd7f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:45:58.748761 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-83a8744d-f33f-4d05-a9ef-e722f574cd7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '967192ce42d64f9b886a6e5d7b194283', 'request_id': 'req-83a8744d-f33f-4d05-a9ef-e722f574cd7f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:45:58.754554 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-83a8744d-f33f-4d05-a9ef-e722f574cd7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '967192ce42d64f9b886a6e5d7b194283', 'request_id': 'req-83a8744d-f33f-4d05-a9ef-e722f574cd7f'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:45:58.766404 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-83a8744d-f33f-4d05-a9ef-e722f574cd7f admin admin] The token cache is being invalidate because project 967192ce42d64f9b886a6e5d7b194283 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:45:58.766693 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-83a8744d-f33f-4d05-a9ef-e722f574cd7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:45:58.767314 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-83a8744d-f33f-4d05-a9ef-e722f574cd7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:45:58.768769 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1450/2923] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:45:58 2022] DELETE /identity/v3/projects/967192ce42d64f9b886a6e5d7b194283 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:46:08.435900 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:46:08.763441 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:46:17.108259 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d05a5-7882-4353-8e4c-38bb02b145cd None None] REQUEST_METHOD: `POST` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:46:17.108811 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d05a5-7882-4353-8e4c-38bb02b145cd None None] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:46:17.108811 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d05a5-7882-4353-8e4c-38bb02b145cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:46:17.128205 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.auth.core [None req-b33d05a5-7882-4353-8e4c-38bb02b145cd None None] MFA Rules not processed for user `0561a33fd6984118b1fd221f9eaaeef6`. Rule list: `[]` (Enabled: `True`). {{(pid=81420) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 10:46:17.129445 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-b33d05a5-7882-4353-8e4c-38bb02b145cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:46:17.183626 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1474/2924] 10.222.0.49 () {58 vars in 958 bytes} [Mon Sep 26 10:46:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 10:46:17.500686 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8744d-f33f-4d05-a9ef-e722f574cd7f admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:46:17.502150 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-83a8744d-f33f-4d05-a9ef-e722f574cd7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:46:17.558696 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8744d-f33f-4d05-a9ef-e722f574cd7f admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:46:17.562246 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530df4b1-2aa6-41d2-b6f0-a616c8f88409 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:46:17.562838 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530df4b1-2aa6-41d2-b6f0-a616c8f88409 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:46:17.563115 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530df4b1-2aa6-41d2-b6f0-a616c8f88409 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:46:17.563382 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530df4b1-2aa6-41d2-b6f0-a616c8f88409 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:46:17.565058 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-530df4b1-2aa6-41d2-b6f0-a616c8f88409 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:46:17.574369 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530df4b1-2aa6-41d2-b6f0-a616c8f88409 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:46:17.575790 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530df4b1-2aa6-41d2-b6f0-a616c8f88409 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:46:17.583944 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1451/2925] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:46:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:46:27.177145 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=25 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:46:27.580124 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=26 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:48:13.254394 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33d05a5-7882-4353-8e4c-38bb02b145cd None None] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:48:13.265252 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33d05a5-7882-4353-8e4c-38bb02b145cd None None] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:48:13.268849 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c3ea55-36e3-469d-b2ac-43be9bb41f4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:48:13.269440 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3ea55-36e3-469d-b2ac-43be9bb41f4f service neutron] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:48:13.269718 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3ea55-36e3-469d-b2ac-43be9bb41f4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:48:13.269990 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3ea55-36e3-469d-b2ac-43be9bb41f4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:48:13.271718 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-63c3ea55-36e3-469d-b2ac-43be9bb41f4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:48:13.282280 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c3ea55-36e3-469d-b2ac-43be9bb41f4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:48:13.283691 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c3ea55-36e3-469d-b2ac-43be9bb41f4f service neutron] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:48:13.293040 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1475/2926] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:48:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 10:48:23.288546 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=6 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:48:26.703960 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530df4b1-2aa6-41d2-b6f0-a616c8f88409 service neutron] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:48:26.712940 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530df4b1-2aa6-41d2-b6f0-a616c8f88409 service neutron] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:48:26.716769 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e134b-6032-4dc0-ac1d-875aa7ab8f24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f841bf31c43b4333a3d834d38b5197e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:48:26.717315 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e134b-6032-4dc0-ac1d-875aa7ab8f24 service neutron] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:48:26.717597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e134b-6032-4dc0-ac1d-875aa7ab8f24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:48:26.717873 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e134b-6032-4dc0-ac1d-875aa7ab8f24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:48:26.719645 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-3a1e134b-6032-4dc0-ac1d-875aa7ab8f24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:48:26.782771 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1e134b-6032-4dc0-ac1d-875aa7ab8f24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:48:26.784202 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1e134b-6032-4dc0-ac1d-875aa7ab8f24 service neutron] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:48:26.794344 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1452/2927] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:48:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:48:29.846588 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c3ea55-36e3-469d-b2ac-43be9bb41f4f service neutron] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:48:29.848198 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-63c3ea55-36e3-469d-b2ac-43be9bb41f4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:48:29.938995 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c3ea55-36e3-469d-b2ac-43be9bb41f4f service neutron] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:48:29.944944 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545097e-3f61-4bdc-9dd4-93df75dbacd9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84b02e7351914cabb67462454b328d05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:48:29.945685 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545097e-3f61-4bdc-9dd4-93df75dbacd9 service placement] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:48:29.946086 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545097e-3f61-4bdc-9dd4-93df75dbacd9 service placement] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:48:29.946443 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545097e-3f61-4bdc-9dd4-93df75dbacd9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:48:29.948707 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-e545097e-3f61-4bdc-9dd4-93df75dbacd9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:48:29.962890 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545097e-3f61-4bdc-9dd4-93df75dbacd9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:48:29.964489 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545097e-3f61-4bdc-9dd4-93df75dbacd9 service placement] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:48:29.976006 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1476/2928] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:48:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:48:32.925971 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e134b-6032-4dc0-ac1d-875aa7ab8f24 service neutron] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:48:32.937701 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e134b-6032-4dc0-ac1d-875aa7ab8f24 service neutron] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:48:32.942885 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9cf8b-ae40-4686-a7af-81f0f374f577 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84b02e7351914cabb67462454b328d05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:48:32.943514 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9cf8b-ae40-4686-a7af-81f0f374f577 service placement] REQUEST_METHOD: `GET` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:48:32.943791 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9cf8b-ae40-4686-a7af-81f0f374f577 service placement] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:48:32.944088 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9cf8b-ae40-4686-a7af-81f0f374f577 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:48:32.945812 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-92d9cf8b-ae40-4686-a7af-81f0f374f577 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:48:32.957498 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9cf8b-ae40-4686-a7af-81f0f374f577 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:48:32.959103 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9cf8b-ae40-4686-a7af-81f0f374f577 service placement] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:48:32.970824 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1453/2929] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:48:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:48:32.996124 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545097e-3f61-4bdc-9dd4-93df75dbacd9 service placement] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:48:32.997661 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-e545097e-3f61-4bdc-9dd4-93df75dbacd9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:48:33.015696 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545097e-3f61-4bdc-9dd4-93df75dbacd9 service placement] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:48:33.021193 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f751193-c556-4d0b-a5c1-f6d782b6df79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:48:33.021841 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f751193-c556-4d0b-a5c1-f6d782b6df79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:48:33.022189 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f751193-c556-4d0b-a5c1-f6d782b6df79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:48:33.022467 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f751193-c556-4d0b-a5c1-f6d782b6df79 admin admin] PATH_INFO: `/v3/users/0561a33fd6984118b1fd221f9eaaeef6` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:48:33.023600 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f751193-c556-4d0b-a5c1-f6d782b6df79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0561a33fd6984118b1fd221f9eaaeef6)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:48:33.025834 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f751193-c556-4d0b-a5c1-f6d782b6df79 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:48:33.075288 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-5f751193-c556-4d0b-a5c1-f6d782b6df79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0561a33fd6984118b1fd221f9eaaeef6'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:48:33.075881 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-5f751193-c556-4d0b-a5c1-f6d782b6df79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0561a33fd6984118b1fd221f9eaaeef6'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:48:33.082750 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-5f751193-c556-4d0b-a5c1-f6d782b6df79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0561a33fd6984118b1fd221f9eaaeef6'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:48:33.095181 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-5f751193-c556-4d0b-a5c1-f6d782b6df79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0561a33fd6984118b1fd221f9eaaeef6'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:48:33.095869 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-5f751193-c556-4d0b-a5c1-f6d782b6df79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0561a33fd6984118b1fd221f9eaaeef6'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:48:33.109532 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1477/2930] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:48:32 2022] DELETE /identity/v3/users/0561a33fd6984118b1fd221f9eaaeef6 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:48:33.275968 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9cf8b-ae40-4686-a7af-81f0f374f577 service placement] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:48:33.287277 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9cf8b-ae40-4686-a7af-81f0f374f577 service placement] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:48:33.348613 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3e00d-ad67-4f05-b29d-43a641436963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:48:33.349251 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3e00d-ad67-4f05-b29d-43a641436963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:48:33.349528 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3e00d-ad67-4f05-b29d-43a641436963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:48:33.349801 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3e00d-ad67-4f05-b29d-43a641436963 admin admin] PATH_INFO: `/v3/projects/1e0f8c6af4d3456c9d700088cd8f5eb5` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:48:33.351615 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3e00d-ad67-4f05-b29d-43a641436963 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e0f8c6af4d3456c9d700088cd8f5eb5)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:48:33.353597 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3e00d-ad67-4f05-b29d-43a641436963 admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:48:33.401344 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-d0d3e00d-ad67-4f05-b29d-43a641436963 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e0f8c6af4d3456c9d700088cd8f5eb5', 'request_id': 'req-d0d3e00d-ad67-4f05-b29d-43a641436963'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:48:33.407444 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-d0d3e00d-ad67-4f05-b29d-43a641436963 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e0f8c6af4d3456c9d700088cd8f5eb5', 'request_id': 'req-d0d3e00d-ad67-4f05-b29d-43a641436963'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:48:33.413530 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-d0d3e00d-ad67-4f05-b29d-43a641436963 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e0f8c6af4d3456c9d700088cd8f5eb5', 'request_id': 'req-d0d3e00d-ad67-4f05-b29d-43a641436963'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:48:33.426618 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-d0d3e00d-ad67-4f05-b29d-43a641436963 admin admin] The token cache is being invalidate because project 1e0f8c6af4d3456c9d700088cd8f5eb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:48:33.426904 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-d0d3e00d-ad67-4f05-b29d-43a641436963 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:48:33.427531 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-d0d3e00d-ad67-4f05-b29d-43a641436963 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:48:33.429008 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1454/2931] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:48:33 2022] DELETE /identity/v3/projects/1e0f8c6af4d3456c9d700088cd8f5eb5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:48:33.433024 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f751193-c556-4d0b-a5c1-f6d782b6df79 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:48:33.434453 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-5f751193-c556-4d0b-a5c1-f6d782b6df79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:48:33.447347 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f751193-c556-4d0b-a5c1-f6d782b6df79 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:48:33.504080 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebad5c9-b3ab-4263-815b-6b9bf5806f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:48:33.504628 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebad5c9-b3ab-4263-815b-6b9bf5806f56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:48:33.504905 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebad5c9-b3ab-4263-815b-6b9bf5806f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:48:33.505177 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebad5c9-b3ab-4263-815b-6b9bf5806f56 admin admin] PATH_INFO: `/v3/users/d3a51714e44e45e9bd5d49a65660ac45` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:48:33.506329 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebad5c9-b3ab-4263-815b-6b9bf5806f56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3a51714e44e45e9bd5d49a65660ac45)` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:48:33.508817 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebad5c9-b3ab-4263-815b-6b9bf5806f56 admin admin] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:48:33.511050 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:48:33.560306 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-4ebad5c9-b3ab-4263-815b-6b9bf5806f56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3a51714e44e45e9bd5d49a65660ac45'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:48:33.560959 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-4ebad5c9-b3ab-4263-815b-6b9bf5806f56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3a51714e44e45e9bd5d49a65660ac45'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:48:33.568120 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-4ebad5c9-b3ab-4263-815b-6b9bf5806f56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3a51714e44e45e9bd5d49a65660ac45'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:48:33.583092 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-4ebad5c9-b3ab-4263-815b-6b9bf5806f56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3a51714e44e45e9bd5d49a65660ac45'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:48:33.583773 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.notifications [None req-4ebad5c9-b3ab-4263-815b-6b9bf5806f56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3a51714e44e45e9bd5d49a65660ac45'} {{(pid=81420) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:48:33.596822 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1478/2932] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Sep 26 10:48:33 2022] DELETE /identity/v3/users/d3a51714e44e45e9bd5d49a65660ac45 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:48:33.769071 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3e00d-ad67-4f05-b29d-43a641436963 admin admin] Authenticating user token {{(pid=81421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:48:33.770600 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.fernet_utils [None req-d0d3e00d-ad67-4f05-b29d-43a641436963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:48:33.787453 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3e00d-ad67-4f05-b29d-43a641436963 admin admin] Validating token access rules against request {{(pid=81421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:48:33.795785 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:48:33.849360 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58cc779-414d-4500-a813-f5f25202751b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ecff1fbd79248b7bcd1e69035be3515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cef67c15d1f4b9695af7c22c68b5822', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:48:33.850151 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58cc779-414d-4500-a813-f5f25202751b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:48:33.850704 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58cc779-414d-4500-a813-f5f25202751b admin admin] SCRIPT_NAME: `/identity` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:48:33.851017 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58cc779-414d-4500-a813-f5f25202751b admin admin] PATH_INFO: `/v3/projects/e9f6ccbccd7a43b5bc1212cc59699e8d` {{(pid=81421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:48:33.852302 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58cc779-414d-4500-a813-f5f25202751b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9f6ccbccd7a43b5bc1212cc59699e8d)` {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:48:33.855097 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58cc779-414d-4500-a813-f5f25202751b admin admin] RBAC: Authorization granted {{(pid=81421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:48:33.900736 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-d58cc779-414d-4500-a813-f5f25202751b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9f6ccbccd7a43b5bc1212cc59699e8d', 'request_id': 'req-d58cc779-414d-4500-a813-f5f25202751b'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:48:33.906731 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-d58cc779-414d-4500-a813-f5f25202751b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9f6ccbccd7a43b5bc1212cc59699e8d', 'request_id': 'req-d58cc779-414d-4500-a813-f5f25202751b'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:48:33.912135 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-d58cc779-414d-4500-a813-f5f25202751b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9f6ccbccd7a43b5bc1212cc59699e8d', 'request_id': 'req-d58cc779-414d-4500-a813-f5f25202751b'} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:48:33.924135 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-d58cc779-414d-4500-a813-f5f25202751b admin admin] The token cache is being invalidate because project e9f6ccbccd7a43b5bc1212cc59699e8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 10:48:33.924418 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-d58cc779-414d-4500-a813-f5f25202751b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:48:33.925044 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG keystone.notifications [None req-d58cc779-414d-4500-a813-f5f25202751b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 10:48:33.926563 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: [pid: 81421|app: 0|req: 1455/2933] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Sep 26 10:48:33 2022] DELETE /identity/v3/projects/e9f6ccbccd7a43b5bc1212cc59699e8d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 10:48:43.593397 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=25,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:48:43.921275 dv-cinder-844563-7-iscsi devstack@keystone.service[81421]: DEBUG dbcounter [-] [81421] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 10:51:01.349671 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebad5c9-b3ab-4263-815b-6b9bf5806f56 admin admin] Authenticating user token {{(pid=81420) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 10:51:01.351357 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-4ebad5c9-b3ab-4263-815b-6b9bf5806f56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:51:01.489107 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebad5c9-b3ab-4263-815b-6b9bf5806f56 admin admin] Validating token access rules against request {{(pid=81420) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 10:51:01.547653 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b7b8b-5da3-4399-b9d3-eff367f0de7e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84b02e7351914cabb67462454b328d05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cff3a6011f4c4893bddffdad3d69f3df', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81420) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 10:51:01.548260 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b7b8b-5da3-4399-b9d3-eff367f0de7e service placement] REQUEST_METHOD: `GET` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 10:51:01.548534 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b7b8b-5da3-4399-b9d3-eff367f0de7e service placement] SCRIPT_NAME: `/identity` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 10:51:01.548801 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b7b8b-5da3-4399-b9d3-eff367f0de7e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81420) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 10:51:01.550573 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.fernet_utils [None req-ba4b7b8b-5da3-4399-b9d3-eff367f0de7e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81420) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 10:51:01.562487 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b7b8b-5da3-4399-b9d3-eff367f0de7e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 10:51:01.563991 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b7b8b-5da3-4399-b9d3-eff367f0de7e service placement] RBAC: Authorization granted {{(pid=81420) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 10:51:01.573840 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: [pid: 81420|app: 0|req: 1479/2934] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Sep 26 10:51:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 10:51:11.569506 dv-cinder-844563-7-iscsi devstack@keystone.service[81420]: DEBUG dbcounter [-] [81420] Writing DB stats keystone:SELECT=89 {{(pid=81420) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}